[SPARK-29320][TESTS] Compare sql/core module in JDK8/11 (Part 1)
#26003
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,143 @@ | ||
| ================================================================================================ | ||
| aggregate without grouping | ||
| ================================================================================================ | ||
|
|
||
| OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 | ||
| Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz | ||
| agg w/o group: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative | ||
| ------------------------------------------------------------------------------------------------------------------------ | ||
| agg w/o group wholestage off 55644 59484 NaN 37.7 26.5 1.0X | ||
|
Member
There was a problem hiding this comment.
Member
Author
There was a problem hiding this comment. Hmm. I'm not sure, but it's the generated result of the current benchmark suite.
Member
There was a problem hiding this comment. This would suggest only 1 trial was run on this one. Not wrong per se but ideally these benchmarks all run a few iterations. |
||
| agg w/o group wholestage on 896 906 8 2340.7 0.4 62.1X | ||
|
|
||
|
|
||
| ================================================================================================ | ||
| stat functions | ||
| ================================================================================================ | ||
|
|
||
| OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 | ||
| Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz | ||
| stddev: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative | ||
| ------------------------------------------------------------------------------------------------------------------------ | ||
| stddev wholestage off 8655 9022 519 12.1 82.5 1.0X | ||
| stddev wholestage on 1306 1323 13 80.3 12.5 6.6X | ||
|
|
||
| OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 | ||
| Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz | ||
| kurtosis: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative | ||
| ------------------------------------------------------------------------------------------------------------------------ | ||
| kurtosis wholestage off 40795 41041 349 2.6 389.0 1.0X | ||
| kurtosis wholestage on 1441 1468 22 72.8 13.7 28.3X | ||
|
|
||
|
|
||
| ================================================================================================ | ||
| aggregate with linear keys | ||
| ================================================================================================ | ||
|
|
||
| OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 | ||
| Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz | ||
| Aggregate w keys: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative | ||
| ------------------------------------------------------------------------------------------------------------------------ | ||
| codegen = F 10559 10956 562 7.9 125.9 1.0X | ||
| codegen = T hashmap = F 6533 6567 34 12.8 77.9 1.6X | ||
| codegen = T hashmap = T 1362 1377 22 61.6 16.2 7.8X | ||
|
|
||
|
|
||
| ================================================================================================ | ||
| aggregate with randomized keys | ||
| ================================================================================================ | ||
|
|
||
| OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 | ||
| Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz | ||
| Aggregate w keys: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative | ||
| ------------------------------------------------------------------------------------------------------------------------ | ||
| codegen = F 12631 12634 3 6.6 150.6 1.0X | ||
| codegen = T hashmap = F 8434 8478 44 9.9 100.5 1.5X | ||
| codegen = T hashmap = T 2484 2598 117 33.8 29.6 5.1X | ||
|
|
||
|
|
||
| ================================================================================================ | ||
| aggregate with string key | ||
| ================================================================================================ | ||
|
|
||
| OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 | ||
| Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz | ||
| Aggregate w string key: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative | ||
| ------------------------------------------------------------------------------------------------------------------------ | ||
| codegen = F 4173 4204 44 5.0 199.0 1.0X | ||
| codegen = T hashmap = F 2664 2721 81 7.9 127.0 1.6X | ||
| codegen = T hashmap = T 1178 1219 59 17.8 56.2 3.5X | ||
|
|
||
|
|
||
| ================================================================================================ | ||
| aggregate with decimal key | ||
| ================================================================================================ | ||
|
|
||
| OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 | ||
| Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz | ||
| Aggregate w decimal key: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative | ||
| ------------------------------------------------------------------------------------------------------------------------ | ||
| codegen = F 3740 3747 9 5.6 178.3 1.0X | ||
| codegen = T hashmap = F 2398 2528 184 8.7 114.3 1.6X | ||
| codegen = T hashmap = T 638 644 7 32.9 30.4 5.9X | ||
|
|
||
|
|
||
| ================================================================================================ | ||
| aggregate with multiple key types | ||
| ================================================================================================ | ||
|
|
||
| OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 | ||
| Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz | ||
| Aggregate w multiple keys: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative | ||
| ------------------------------------------------------------------------------------------------------------------------ | ||
| codegen = F 6874 6896 31 3.1 327.8 1.0X | ||
| codegen = T hashmap = F 3866 3886 28 5.4 184.3 1.8X | ||
| codegen = T hashmap = T 2619 2641 31 8.0 124.9 2.6X | ||
|
|
||
|
|
||
| ================================================================================================ | ||
| max function bytecode size of wholestagecodegen | ||
| ================================================================================================ | ||
|
|
||
| OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 | ||
| Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz | ||
| max function bytecode size: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative | ||
| ------------------------------------------------------------------------------------------------------------------------ | ||
| codegen = F 793 806 19 0.8 1209.7 1.0X | ||
| codegen = T hugeMethodLimit = 10000 401 456 61 1.6 611.2 2.0X | ||
| codegen = T hugeMethodLimit = 1500 694 715 19 0.9 1059.3 1.1X | ||
|
|
||
|
|
||
| ================================================================================================ | ||
| cube | ||
| ================================================================================================ | ||
|
|
||
| OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 | ||
| Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz | ||
| cube: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative | ||
| ------------------------------------------------------------------------------------------------------------------------ | ||
| cube wholestage off 3616 3630 19 1.4 689.8 1.0X | ||
| cube wholestage on 1819 1866 41 2.9 347.0 2.0X | ||
|
|
||
|
|
||
| ================================================================================================ | ||
| hash and BytesToBytesMap | ||
| ================================================================================================ | ||
|
|
||
| OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 | ||
| Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz | ||
| BytesToBytesMap: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative | ||
| ------------------------------------------------------------------------------------------------------------------------ | ||
| UnsafeRowhash 321 321 1 65.4 15.3 1.0X | ||
| murmur3 hash 145 145 0 144.5 6.9 2.2X | ||
| fast hash 70 71 1 298.9 3.3 4.6X | ||
| arrayEqual 198 199 2 105.7 9.5 1.6X | ||
| Java HashMap (Long) 132 136 3 158.6 6.3 2.4X | ||
| Java HashMap (two ints) 152 156 4 138.3 7.2 2.1X | ||
| Java HashMap (UnsafeRow) 819 839 17 25.6 39.1 0.4X | ||
| LongToUnsafeRowMap (opt=false) 459 472 25 45.7 21.9 0.7X | ||
| LongToUnsafeRowMap (opt=true) 107 108 1 195.5 5.1 3.0X | ||
| BytesToBytesMap (off Heap) 1012 1019 10 20.7 48.3 0.3X | ||
| BytesToBytesMap (on Heap) 963 974 18 21.8 45.9 0.3X | ||
| Aggregate HashMap 41 43 2 515.1 1.9 7.9X | ||
|
|
||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just in case,
3.10kernel (if it is version of Linux kernel) is 2 years old. Are we sure we are interested in results on this old version?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MaxGekk . This is Java comparison, not OS comparison. We didn't care Mac OS benchmark result either, do we?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the point is that the benchmark ratios could be different on another OS, but, I think this is fine. Eventually we can try benchmarking consistently on a later distro / 4.x kernel