[SPARK-53524][CONNECT][SQL] Fix temporal value conversion in LiteralValueProtoConverter

[heyihong]

What changes were proposed in this pull request?

This PR fixes temporal value conversion issues in the LiteralValueProtoConverter for Spark Connect. The main changes include:

1. Fixed temporal value conversion in getConverter method: Updated the conversion logic for temporal data types (DATE, TIMESTAMP, TIMESTAMP_NTZ, DAY_TIME_INTERVAL, YEAR_MONTH_INTERVAL, TIME) to use proper utility methods from SparkDateTimeUtils and SparkIntervalUtils instead of directly returning raw protobuf values.

2. Added comprehensive test coverage: Extended the PlanGenerationTestSuite with a new test case that includes a tuple containing all temporal types to ensure proper conversion and serialization.

3. Updated test expectations: Modified the expected explain output and query test files to reflect the corrected temporal value handling.

Why are the changes needed?

The struct type in typedlit doesn't work well with temporal values due to bugs in type conversions. For example, the code below fails:

import org.apache.spark.sql.functions.typedlit

spark.sql("select 1").select(typedlit((1, java.time.LocalDate.of(2020, 10, 10)))).collect()
"""
org.apache.spark.SparkIllegalArgumentException: The value (18545) of the type (java.lang.Integer) cannot be converted to the DATE type.
  org.apache.spark.sql.catalyst.CatalystTypeConverters$DateConverter$.toCatalystImpl(CatalystTypeConverters.scala:356)
  org.apache.spark.sql.catalyst.CatalystTypeConverters$DateConverter$.toCatalystImpl(CatalystTypeConverters.scala:347)
  org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:110)
  org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:271)
  org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:251)
  org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:110)
  org.apache.spark.sql.catalyst.CatalystTypeConverters$.$anonfun$createToCatalystConverter$2(CatalystTypeConverters.scala:532)
  org.apache.spark.sql.connect.planner.LiteralExpressionProtoConverter$.toCatalystExpression(LiteralExpressionProtoConverter.scala:116)
"""

Does this PR introduce any user-facing change?

Yes. This PR fixes temporal value conversion in LiteralValueProtoConverter, allowing the struct type in typedlit to work with temporal values.

How was this patch tested?

build/sbt "connect-client-jvm/testOnly org.apache.spark.sql.PlanGenerationTestSuite"
build/sbt "connect/testOnly org.apache.spark.sql.connect.ProtoToParsedPlanTestSuite"

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Cursor 1.5.11

[heyihong]

@cloud-fan @hvanhovell @zhengruifeng

[cloud-fan]

I need a bit more context to understand it better. So at the client side, fn.typedLit takes java classes like LocalDate, and convert it to int before putting it into the protobuf message. At the server side, v.getDate returns int and we need to convert it back to the LocalDate.

This looks a bit messy to me. If the client side already knows how to convert external java objects to internal catalyst values, why does server need this conversion?

[heyihong]

@cloud-fan I am not sure if the client side knows how to convert external Java objects into internal Catalyst values. The conversion occurs here in LiteralValueProtoConverter and the server just converts the Proto value back to the Java object.

This code path is also needed in the Scala client, since the client may need to convert temporal values from proto observed metrics received from the server into Java objects.

[cloud-fan]

let's rename def getConverter to getScalaConverter

[cloud-fan]

thanks, merging to master!

[cloud-fan]

@heyihong can you open a backport PR for 4.0?