Skip to content

[SPARK-15613] [SQL] Fix incorrect days to millis conversion due to Daylight Saving Time #13652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
davies wants to merge 5 commits into from
Closed

[SPARK-15613] [SQL] Fix incorrect days to millis conversion due to Daylight Saving Time #13652

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
20904c4
fix toUTCTime and daysToMillis
Jun 13, 2016
ac3d2b5
cover all timezone
Jun 14, 2016
f23b545
more acurate
Jun 15, 2016
9df8c84
skip some days in tests
Jun 16, 2016
88df176
fallback to java.util.Date
Jun 18, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
more acurate
  • Loading branch information
Davies Liu committed Jun 15, 2016
commit f23b54563f488c7654b6cb8911daf03a28d1bb95
15 changes: 10 additions & 5 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -857,13 +857,18 @@ object DateTimeUtils {
var guess = tz.getRawOffset
// the actual offset should be calculated based on milliseconds in UTC
var actual = tz.getOffset(millisLocal - guess)
// At the start of DST, the local time is forwarded by an hour, so it's OK to get the
// local time bigger than current one after an round trip (local -> UTC -> local).
// At the end of DST, the local time is backwarded by an hour, actual offset will be
// less than guess, we should decrease the guess and try again.
while (actual < guess) {
if (actual != guess) {
// try with the result as guess
guess = actual
actual = tz.getOffset(millisLocal - guess)
// At the start of DST, the local time is forwarded by an hour, so it's OK to get the
// local time bigger than current one after an round trip (local -> UTC -> local).
// At the end of DST, the local time is backwarded by an hour, actual offset will be
// less than guess, we should decrease the guess and try again.
while (actual < guess) {
guess = actual
actual = tz.getOffset(millisLocal - guess)
}
}
guess
}
Expand Down