[SPARK-50364][SQL] Implement serialization for LocalDateTime type in Row.jsonValue

[karim-ramadan]

What changes were proposed in this pull request?

In this Pull request, I propose to add a LocalDateTime serializer to the Row.jsonValue method

    case (d: LocalDateTime, _) => JString(timestampFormatter.format(d))

In order to enable JSON serialization of TimestampNTZType columns

Why are the changes needed?

Currently trying to serialize a Row containing a TimestampNTZType column results in an error:

[FAILED_ROW_TO_JSON] Failed to convert the row value '2018-05-14T12:13' of the class class java.time.LocalDateTime to the target SQL type "TIMESTAMPNTZTYPE" in the JSON format. SQLSTATE: 2203G
org.apache.spark.SparkIllegalArgumentException: [FAILED_ROW_TO_JSON] Failed to convert the row value '2018-05-14T12:13' of the class class java.time.LocalDateTime to the target SQL type "TIMESTAMPNTZTYPE" in the JSON format. SQLSTATE: 2203G
	at org.apache.spark.sql.Row.toJson$1(Row.scala:663)
	at org.apache.spark.sql.Row.toJson$1(Row.scala:651)
	at org.apache.spark.sql.Row.jsonValue(Row.scala:665)
	at org.apache.spark.sql.Row.jsonValue$(Row.scala:598)
	at org.apache.spark.sql.catalyst.expressions.GenericRow.jsonValue(rows.scala:28)
	at org.apache.spark.sql.RowJsonSuite.$anonfun$testJson$1(RowJsonSuite.scala:41)

How to reproduce the issue:

import org.apache.spark.sql.Row
import java.time.LocalDateTime
val r = Row.fromSeq(LocalDateTime.of(2024,8,10,12,33) :: Nil)
r.json
r.prettyJson

Does this PR introduce any user-facing change?

No

How was this patch tested?

Tests were added to the already existing RowJsonSuite.scala class

Was this patch authored or co-authored using generative AI tooling?

No

[karim-ramadan]

Related issue SPARK-50364

[MaxGekk]

+1, LGTM. All GAs passed. Merging to master.
Thank you, @karim-ramadan.

[MaxGekk]

@karim-ramadan Congratulations with your first contribution to Apache Spark!

[karim-ramadan]

@MaxGekk thanks for the fast response and review