Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
8080c93
[PYSPARK] Updates to Accumulators
LucaCanali Jul 18, 2018
14b50d7
[SPARK-24987][SS] - Fix Kafka consumer leak when no new offsets for T…
Aug 4, 2018
136588e
[SPARK-25015][BUILD] Update Hadoop 2.7 to 2.7.7
srowen Aug 4, 2018
597a77e
Merge branch 'branch-2.3' of github.com:apache/spark into branch-2.3-…
markhamstra Aug 7, 2018
9fb70f4
[SPARK-24948][SHS][BACKPORT-2.3] Delegate check access permissions to…
mgaido91 Aug 8, 2018
7d465d8
[MINOR][BUILD] Update Jetty to 9.3.24.v20180605
srowen Aug 9, 2018
9bfc55b
[SPARK-25076][SQL] SQLConf should not be retrieved from a stopped Spa…
cloud-fan Aug 9, 2018
b426ec5
[SPARK-24950][SQL] DateTimeUtilsSuite daysToMillis and millisToDays f…
d80tb7 Jul 28, 2018
6930f48
Preparing Spark release v2.3.2-rc4
jerryshao Aug 10, 2018
e66f3f9
Preparing development version 2.3.3-SNAPSHOT
jerryshao Aug 10, 2018
7306ac7
[MINOR][BUILD] Add ECCN notice required by http://www.apache.org/dev/…
srowen Aug 10, 2018
04c6520
[SPARK-25081][CORE] Nested spill in ShuffleExternalSorter should not …
zsxwing Aug 10, 2018
a0a7e41
[SPARK-24908][R][STYLE] removing spaces to make lintr happy
shaneknapp Jul 24, 2018
b9b35b9
[SPARK-25084][SQL][BACKPORT-2.3] distribute by" on multiple columns (…
LantaoJin Aug 13, 2018
787790b
[SPARK-25028][SQL] Avoid NPE when analyzing partition with NULL values
mgaido91 Aug 13, 2018
4dc8225
Preparing Spark release v2.3.2-rc5
jerryshao Aug 14, 2018
29a0403
Preparing development version 2.3.3-SNAPSHOT
jerryshao Aug 14, 2018
0856b82
[MINOR][SQL][DOC] Fix `to_json` example in function description and doc
dongjoon-hyun Aug 14, 2018
34191e6
[SPARK-25051][SQL] FixNullability should not stop on AnalysisBarrier
mgaido91 Aug 14, 2018
032f6d9
[MINOR][DOC][SQL] use one line for annotation arg value
mengxr Aug 18, 2018
ea01e36
[SPARK-25144][SQL][TEST][BRANCH-2.3] Free aggregate map when task ends
cloud-fan Aug 20, 2018
9702bb6
[DOCS] Fixed NDCG formula issues
yueguoguo Aug 20, 2018
8bde467
[SPARK-25114][CORE] Fix RecordBinaryComparator when subtraction betwe…
jiangxb1987 Aug 21, 2018
9cb9d72
[SPARK-25114][2.3][CORE][FOLLOWUP] Fix RecordBinaryComparatorSuite bu…
jiangxb1987 Aug 21, 2018
fcc9bd6
[SPARK-25205][CORE] Fix typo in spark.network.crypto.keyFactoryIterat…
squito Aug 24, 2018
42c1fdd
[SPARK-25234][SPARKR] avoid integer overflow in parallelize
mengxr Aug 24, 2018
f598382
[SPARK-25124][ML] VectorSizeHint setSize and getSize don't return val…
huaxingao Aug 24, 2018
67ff50d
Merge branch 'branch-2.3' of github.com:apache/spark into branch-2.3-…
markhamstra Aug 27, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[SPARK-24950][SQL] DateTimeUtilsSuite daysToMillis and millisToDays f…
…ails w/java 8 181-b13

## What changes were proposed in this pull request?

- Update DateTimeUtilsSuite so that when testing roundtripping in daysToMillis and millisToDays multiple skipdates can be specified.
- Updated test so that both new years eve 2014 and new years day 2015 are skipped for kiribati time zones.  This is necessary as java versions pre 181-b13 considered new years day 2015 to be skipped while susequent versions corrected this to new years eve.

## How was this patch tested?
Unit tests

Author: Chris Martin <chris@cmartinit.co.uk>

Closes apache#21901 from d80tb7/SPARK-24950_datetimeUtilsSuite_failures.

(cherry picked from commit c5b8d54)
Signed-off-by: Sean Owen <srowen@gmail.com>
  • Loading branch information
d80tb7 authored and srowen committed Aug 9, 2018
commit b426ec583fb5176461c5b0c7112d2194af66d93d
Original file line number Diff line number Diff line change
Expand Up @@ -650,18 +650,18 @@ class DateTimeUtilsSuite extends SparkFunSuite {
assert(daysToMillis(16800, TimeZoneGMT) === c.getTimeInMillis)

// There are some days are skipped entirely in some timezone, skip them here.
val skipped_days = Map[String, Int](
"Kwajalein" -> 8632,
"Pacific/Apia" -> 15338,
"Pacific/Enderbury" -> 9131,
"Pacific/Fakaofo" -> 15338,
"Pacific/Kiritimati" -> 9131,
"Pacific/Kwajalein" -> 8632,
"MIT" -> 15338)
val skipped_days = Map[String, Set[Int]](
"Kwajalein" -> Set(8632),
"Pacific/Apia" -> Set(15338),
"Pacific/Enderbury" -> Set(9130, 9131),
"Pacific/Fakaofo" -> Set(15338),
"Pacific/Kiritimati" -> Set(9130, 9131),
"Pacific/Kwajalein" -> Set(8632),
"MIT" -> Set(15338))
for (tz <- DateTimeTestUtils.ALL_TIMEZONES) {
val skipped = skipped_days.getOrElse(tz.getID, Int.MinValue)
val skipped = skipped_days.getOrElse(tz.getID, Set.empty)
(-20000 to 20000).foreach { d =>
if (d != skipped) {
if (!skipped.contains(d)) {
assert(millisToDays(daysToMillis(d, tz), tz) === d,
s"Round trip of ${d} did not work in tz ${tz}")
}
Expand Down