[SPARK-35220][SQL] DayTimeIntervalType/YearMonthIntervalType show different between Hive SerDe and row format delimited

[AngersZhuuuu]

What changes were proposed in this pull request?

DayTimeIntervalType/YearMonthIntervalString show different between Hive SerDe and row format delimited.
Create this pr to add a test and have disscuss.

For this problem I think we have two direction:

1. leave it as current and add a item t explain this in migration guide docs.
2. Since we should not change hive serde's behavior, so we can cast spark row format delimited's behavior to use cast DayTimeIntervalType/YearMonthIntervalType as HIVE_STYLE

Why are the changes needed?

Add UT

Does this PR introduce any user-facing change?

No

How was this patch tested?

added ut

[AngersZhuuuu]

Gentle ping @cloud-fan @MaxGekk @maropu

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42441/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42441/

[SparkQA]

Test build #137920 has finished for PR 32335 at commit ab94c21.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #137921 has finished for PR 32335 at commit 9f1bd02.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

Merged to master.

[AngersZhuuuu]

@HyukjinKwon Since you just merged this, I think I need to add a follow up one to add this behavior in migration guide? ok?

[HyukjinKwon]

please go ahead.

[cloud-fan]

so the spark-sql shell and df.show have different formats for intervals?

[AngersZhuuuu]

so the spark-sql shell and df.show have different formats for intervals?

Yea, have this problem too, since spark sql follow hive format. What should I to do next?

[cloud-fan]

Is interval format the only difference between hive format and spark cast?

[AngersZhuuuu]

Is interval format the only difference between hive format and spark cast?

Yea， ANSI_STYLE and HIVE_STYLE

[cloud-fan]

Maybe we should have a new Expression ToHiveString and use it in df.show and TRANSFORM, so that they are consistent.

[AngersZhuuuu]

Maybe we should have a new Expression ToHiveString and use it in df.show and TRANSFORM, so that they are consistent.

Yea, create a ticket https://issues.apache.org/jira/browse/SPARK-35228