Skip to content

[SPARK-21739][SQL]Cast expression should initialize timezoneId when it is called statically to convert something into TimestampType #18960

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
DonnyZone wants to merge 7 commits into from
Closed

[SPARK-21739][SQL]Cast expression should initialize timezoneId when it is called statically to convert something into TimestampType #18960

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
2537c1e
spark-21739
DonnyZone Aug 16, 2017
86331e3
fix code style
DonnyZone Aug 16, 2017
492b756
correct answer
DonnyZone Aug 16, 2017
a264e3a
Obtain SQLConf from sparksession
DonnyZone Aug 17, 2017
5a04ddc
fix test case queries
DonnyZone Aug 17, 2017
93581bb
remove a blank line
DonnyZone Aug 17, 2017
5b051dd
refractor implementation
DonnyZone Aug 18, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Obtain SQLConf from sparksession
  • Loading branch information
@DonnyZone
DonnyZone committed Aug 17, 2017
commit a264e3aa166d2e83832a82489669893f41ff9749
3 changes: 1 addition & 2 deletions sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.catalyst.InternalRow
import org.apache.spark.sql.catalyst.expressions._
import org.apache.spark.sql.catalyst.util.DateTimeUtils
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.unsafe.types.UTF8String
import org.apache.spark.util.{SerializableConfiguration, Utils}

Expand Down Expand Up @@ -229,7 +228,7 @@ class HadoopTableReader(
partitionKeyAttrs.foreach { case (attr, ordinal) =>
val partOrdinal = partitionKeys.indexOf(attr)
row(ordinal) = Cast(Literal(rawPartValues(partOrdinal)), attr.dataType,
Option(SQLConf.get.sessionLocalTimeZone)).eval(null)
Option(sparkSession.sessionState.conf.sessionLocalTimeZone)).eval(null)
}
}

Expand Down
4 changes: 2 additions & 2 deletions sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScanExec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ import org.apache.spark.sql.execution._
import org.apache.spark.sql.execution.metric.SQLMetrics
import org.apache.spark.sql.hive._
import org.apache.spark.sql.hive.client.HiveClientImpl
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.types.{BooleanType, DataType}
import org.apache.spark.util.Utils

Expand Down Expand Up @@ -105,7 +104,8 @@ case class HiveTableScanExec(
hadoopConf)

private def castFromString(value: String, dataType: DataType) = {
Cast(Literal(value), dataType, Option(SQLConf.get.sessionLocalTimeZone)).eval(null)
Cast(Literal(value), dataType,
Option(sparkSession.sessionState.conf.sessionLocalTimeZone)).eval(null)
}

private def addColumnMetadataToConf(hiveConf: Configuration): Unit = {
Expand Down