Skip to content

[WIP][SQL] Optimize fromJavaDate() and toJavaDate() #28091

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MaxGekk wants to merge 5 commits into from
Closed

[WIP][SQL] Optimize fromJavaDate() and toJavaDate() #28091

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ebb8d13
Optimize fromJavaDate() and toJavaDate()
MaxGekk Apr 1, 2020
831dfe1
Re-gen DateTimeBenchmark results on JDK 8
MaxGekk Apr 1, 2020
0830158
Re-gen DateTimeBenchmark results on JDK 11
MaxGekk Apr 1, 2020
ee78740
Re-use micros rebasing functions
MaxGekk Apr 1, 2020
586517f
Merge remote-tracking branch 'remotes/origin/master' into optimize-fr…
MaxGekk Apr 9, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 4 additions & 12 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,16 +101,8 @@ object DateTimeUtils {
* @return The number of days since epoch from java.sql.Date.
*/
def fromJavaDate(date: Date): SQLDate = {
val era = if (date.before(julianCommonEraStart)) 0 else 1
val localDate = LocalDate
.of(date.getYear + 1900, date.getMonth + 1, 1)
.`with`(ChronoField.ERA, era)
// Add days separately to convert dates existed in Julian calendar but not
// in Proleptic Gregorian calendar. For example, 1000-02-29 is valid date
// in Julian calendar because 1000 is a leap year but 1000 is not a leap
// year in Proleptic Gregorian calendar. And 1000-02-29 doesn't exist in it.
.plusDays(date.getDate - 1) // Returns the next valid date after `date.getDate - 1` days
localDateToDays(localDate)
val rebasedMicros = rebaseJulianToGregorianMicros(millisToMicros(date.getTime))
microsToDays(rebasedMicros)
}

/**
Expand All @@ -127,8 +119,8 @@ object DateTimeUtils {
* @return A `java.sql.Date` from number of days since epoch.
*/
def toJavaDate(daysSinceEpoch: SQLDate): Date = {
val localDate = LocalDate.ofEpochDay(daysSinceEpoch)
new Date(localDate.getYear - 1900, localDate.getMonthValue - 1, localDate.getDayOfMonth)
val rebasedMicros = rebaseGregorianToJulianMicros(daysToMicros(daysSinceEpoch))
new Date(microsToMillis(rebasedMicros))
}

/**
Expand Down