Skip to content

[SPARK-31426][SQL] Fix perf regressions of toJavaTimestamp/fromJavaTimestamp #28189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MaxGekk wants to merge 8 commits into from
Closed

[SPARK-31426][SQL] Fix perf regressions of toJavaTimestamp/fromJavaTimestamp #28189

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a56c752
Rewrite toJavaTimestamp/fromJavaTimestamp
MaxGekk Apr 11, 2020
794cc5e
Re-gen DateTimeRebaseBenchmark results on JDK 8
MaxGekk Apr 11, 2020
fd06ecc
Re-gen DateTimeBenchmark results on JDK 8
MaxGekk Apr 11, 2020
94dc31e
Re-gen benchmark results on JDK 11
MaxGekk Apr 11, 2020
055ac97
Merge remote-tracking branch 'origin/master' into optimize-to-from-ja…
MaxGekk Apr 13, 2020
b331ad8
Merge remote-tracking branch 'origin/master' into optimize-to-from-ja…
MaxGekk Apr 13, 2020
5dc9a3a
Re-gen DateTimeRebaseBenchmark results on JDK 11
MaxGekk Apr 13, 2020
a28f2a1
Re-gen DateTimeRebaseBenchmark results on JDK 8
MaxGekk Apr 13, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Re-gen DateTimeRebaseBenchmark results on JDK 11
  • Loading branch information
@MaxGekk
MaxGekk committed Apr 13, 2020
commit 5dc9a3a5c06edb3684f4a8a6cfe6427bb1ae9f5c
88 changes: 44 additions & 44 deletions sql/core/benchmarks/DateTimeRebaseBenchmark-jdk11-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,49 +6,49 @@ OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save dates to parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 19946 19946 0 5.0 199.5 1.0X
before 1582, noop 10910 10910 0 9.2 109.1 1.8X
after 1582, rebase off 32065 32065 0 3.1 320.7 0.6X
after 1582, rebase on 31424 31424 0 3.2 314.2 0.6X
before 1582, rebase off 21764 21764 0 4.6 217.6 0.9X
before 1582, rebase on 22700 22700 0 4.4 227.0 0.9X
after 1582, noop 21171 21171 0 4.7 211.7 1.0X
before 1582, noop 11036 11036 0 9.1 110.4 1.9X
after 1582, rebase off 34321 34321 0 2.9 343.2 0.6X
after 1582, rebase on 33269 33269 0 3.0 332.7 0.6X
before 1582, rebase off 22016 22016 0 4.5 220.2 1.0X
before 1582, rebase on 23338 23338 0 4.3 233.4 0.9X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load dates from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off, rebase off 13054 13273 193 7.7 130.5 1.0X
after 1582, vec off, rebase on 13337 13418 117 7.5 133.4 1.0X
after 1582, vec on, rebase off 3666 3740 116 27.3 36.7 3.6X
after 1582, vec on, rebase on 5097 5160 70 19.6 51.0 2.6X
before 1582, vec off, rebase off 12908 12984 76 7.7 129.1 1.0X
before 1582, vec off, rebase on 14514 14575 53 6.9 145.1 0.9X
before 1582, vec on, rebase off 3718 3790 63 26.9 37.2 3.5X
before 1582, vec on, rebase on 5472 5527 48 18.3 54.7 2.4X
after 1582, vec off, rebase off 12791 13089 287 7.8 127.9 1.0X
after 1582, vec off, rebase on 13203 13271 81 7.6 132.0 1.0X
after 1582, vec on, rebase off 3709 3764 49 27.0 37.1 3.4X
after 1582, vec on, rebase on 5082 5114 29 19.7 50.8 2.5X
before 1582, vec off, rebase off 13059 13153 87 7.7 130.6 1.0X
before 1582, vec off, rebase on 14211 14236 27 7.0 142.1 0.9X
before 1582, vec on, rebase off 3687 3749 72 27.1 36.9 3.5X
before 1582, vec on, rebase on 5449 5497 56 18.4 54.5 2.3X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save timestamps to parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 2718 2718 0 36.8 27.2 1.0X
before 1582, noop 2774 2774 0 36.0 27.7 1.0X
after 1582, rebase off 16515 16515 0 6.1 165.2 0.2X
after 1582, rebase on 18619 18619 0 5.4 186.2 0.1X
before 1582, rebase off 16382 16382 0 6.1 163.8 0.2X
before 1582, rebase on 19580 19580 0 5.1 195.8 0.1X
after 1582, noop 2831 2831 0 35.3 28.3 1.0X
before 1582, noop 2816 2816 0 35.5 28.2 1.0X
after 1582, rebase off 15543 15543 0 6.4 155.4 0.2X
after 1582, rebase on 18391 18391 0 5.4 183.9 0.2X
before 1582, rebase off 15747 15747 0 6.4 157.5 0.2X
before 1582, rebase on 18846 18846 0 5.3 188.5 0.2X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load timestamps from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off, rebase off 15082 15112 43 6.6 150.8 1.0X
after 1582, vec off, rebase on 18035 18105 107 5.5 180.3 0.8X
after 1582, vec on, rebase off 4948 4970 19 20.2 49.5 3.0X
after 1582, vec on, rebase on 8761 8780 25 11.4 87.6 1.7X
before 1582, vec off, rebase off 15285 15389 103 6.5 152.8 1.0X
before 1582, vec off, rebase on 18277 18368 80 5.5 182.8 0.8X
before 1582, vec on, rebase off 4938 4954 15 20.3 49.4 3.1X
before 1582, vec on, rebase on 9220 9248 33 10.8 92.2 1.6X
after 1582, vec off, rebase off 16126 16216 78 6.2 161.3 1.0X
after 1582, vec off, rebase on 18277 18453 165 5.5 182.8 0.9X
after 1582, vec on, rebase off 5030 5067 42 19.9 50.3 3.2X
after 1582, vec on, rebase on 8553 8583 43 11.7 85.5 1.9X
before 1582, vec off, rebase off 15828 15872 39 6.3 158.3 1.0X
before 1582, vec off, rebase on 18899 18959 103 5.3 189.0 0.9X
before 1582, vec on, rebase off 4961 5009 43 20.2 49.6 3.3X
before 1582, vec on, rebase on 9099 9140 40 11.0 91.0 1.8X


================================================================================================
Expand All @@ -59,36 +59,36 @@ OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save dates to ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 19820 19820 0 5.0 198.2 1.0X
before 1582, noop 10825 10825 0 9.2 108.2 1.8X
after 1582 27705 27705 0 3.6 277.1 0.7X
before 1582 19390 19390 0 5.2 193.9 1.0X
after 1582, noop 21026 21026 0 4.8 210.3 1.0X
before 1582, noop 11040 11040 0 9.1 110.4 1.9X
after 1582 28171 28171 0 3.5 281.7 0.7X
before 1582 18955 18955 0 5.3 189.5 1.1X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load dates from ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off 10401 10426 33 9.6 104.0 1.0X
after 1582, vec on 3788 3813 29 26.4 37.9 2.7X
before 1582, vec off 10832 10843 11 9.2 108.3 1.0X
before 1582, vec on 4182 4219 46 23.9 41.8 2.5X
after 1582, vec off 10876 10931 49 9.2 108.8 1.0X
after 1582, vec on 3900 3913 20 25.6 39.0 2.8X
before 1582, vec off 11165 11174 12 9.0 111.6 1.0X
before 1582, vec on 4208 4214 7 23.8 42.1 2.6X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save timestamps to ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 2787 2787 0 35.9 27.9 1.0X
before 1582, noop 2777 2777 0 36.0 27.8 1.0X
after 1582 52456 52456 0 1.9 524.6 0.1X
before 1582 54983 54983 0 1.8 549.8 0.1X
after 1582, noop 2924 2924 0 34.2 29.2 1.0X
before 1582, noop 2820 2820 0 35.5 28.2 1.0X
after 1582 22228 22228 0 4.5 222.3 0.1X
before 1582 22590 22590 0 4.4 225.9 0.1X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load timestamps from ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off 38336 38425 88 2.6 383.4 1.0X
after 1582, vec on 30089 30198 96 3.3 300.9 1.3X
before 1582, vec off 42739 42801 87 2.3 427.4 0.9X
before 1582, vec on 31530 31591 61 3.2 315.3 1.2X
after 1582, vec off 13591 13658 59 7.4 135.9 1.0X
after 1582, vec on 7399 7488 126 13.5 74.0 1.8X
before 1582, vec off 14065 14096 30 7.1 140.7 1.0X
before 1582, vec on 7950 8127 249 12.6 79.5 1.7X