Skip to content

[SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark #22617

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
peter-toth wants to merge 9 commits into from
Closed

[SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark #22617

Changes from 2 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
5bc905a
[SPARK-25484][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark
peter-toth Oct 2, 2018
93892de
[SPARK-25484][TEST] some code deduplication, fix slow sbt benchmark run
peter-toth Oct 3, 2018
a26dbc7
[SPARK-25484][SQL][TEST] rebase on master and fix
Oct 14, 2018
0f676bc
[SPARK-25484][SQL][TEST] revert ExternalAppendOnlyUnsafeRowArray init…
peter-toth Oct 30, 2018
0b04fa3
[SPARK-25484][SQL][TEST] rebase and fix build issue
peter-toth Jan 8, 2019
702e61c
[SPARK-25484][SQL][TEST] fix review findings
peter-toth Jan 9, 2019
87c4180
EC2 result
dongjoon-hyun Jan 9, 2019
411174a
Merge pull request #3 from dongjoon-hyun/PR-22617
peter-toth Jan 9, 2019
b0d829e
[SPARK-25484][SQL][TEST] fix comment
peter-toth Jan 9, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 20 additions & 25 deletions sql/core/benchmarks/ExternalAppendOnlyUnsafeRowArrayBenchmark-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,49 +2,44 @@
WITHOUT SPILL
================================================================================================

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz

OpenJDK 64-Bit Server VM 1.8.0_191-b12 on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Array with 100000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
ArrayBuffer 4810 / 5120 21.3 47.0 1.0X
ExternalAppendOnlyUnsafeRowArray 4996 / 5043 20.5 48.8 1.0X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
ArrayBuffer 6378 / 6550 16.1 62.3 1.0X
ExternalAppendOnlyUnsafeRowArray 6196 / 6242 16.5 60.5 1.0X

OpenJDK 64-Bit Server VM 1.8.0_191-b12 on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Array with 1000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
ArrayBuffer 8839 / 8951 29.7 33.7 1.0X
ExternalAppendOnlyUnsafeRowArray 9884 / 9888 26.5 37.7 0.9X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
ArrayBuffer 11988 / 12027 21.9 45.7 1.0X
ExternalAppendOnlyUnsafeRowArray 37480 / 37574 7.0 143.0 0.3X

OpenJDK 64-Bit Server VM 1.8.0_191-b12 on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Array with 30000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
ArrayBuffer 19980 / 20061 24.6 40.6 1.0X
ExternalAppendOnlyUnsafeRowArray 21630 / 21645 22.7 44.0 0.9X
ArrayBuffer 23536 / 23538 20.9 47.9 1.0X
ExternalAppendOnlyUnsafeRowArray 31275 / 31277 15.7 63.6 0.8X


================================================================================================
WITH SPILL
================================================================================================

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz

OpenJDK 64-Bit Server VM 1.8.0_191-b12 on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Spilling with 1000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
UnsafeExternalSorter 15829 / 15845 16.6 60.4 1.0X
ExternalAppendOnlyUnsafeRowArray 10158 / 10174 25.8 38.7 1.6X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
UnsafeExternalSorter 29241 / 29279 9.0 111.5 1.0X
ExternalAppendOnlyUnsafeRowArray 14309 / 14313 18.3 54.6 2.0X

OpenJDK 64-Bit Server VM 1.8.0_191-b12 on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Spilling with 10000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
UnsafeExternalSorter 6 / 6 28.3 35.3 1.0X
ExternalAppendOnlyUnsafeRowArray 6 / 7 27.7 36.1 1.0X
UnsafeExternalSorter 11 / 11 14.8 67.4 1.0X
ExternalAppendOnlyUnsafeRowArray 9 / 9 17.6 56.8 1.2X

Copy link
Member

@dongjoon-hyun dongjoon-hyun Jan 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran the original master branch and get the following. Since the trend is the same, this refactoring PR looks safe.

$ bin/spark-submit --class org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark --jars core/target/scala-2.12/spark-core_2.12-3.0.0-SNAPSHOT-tests.jar sql/core/target/scala-2.12/spark-sql_2.12-3.0.0-SNAPSHOT-tests.jar
...
Array with 1000 rows:                    Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
ArrayBuffer                                   9556 / 9633         27.4          36.5       1.0X
ExternalAppendOnlyUnsafeRowArray            18514 / 18700         14.2          70.6       0.5X

Array with 30000 rows:                   Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
ArrayBuffer                                 22180 / 22195         22.2          45.1       1.0X
ExternalAppendOnlyUnsafeRowArray            24254 / 24331         20.3          49.3       0.9X

Array with 100000 rows:                  Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
ArrayBuffer                                   4998 / 5052         20.5          48.8       1.0X
ExternalAppendOnlyUnsafeRowArray              4778 / 4821         21.4          46.7       1.0X

Spilling with 1000 rows:                 Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
UnsafeExternalSorter                        17536 / 17596         14.9          66.9       1.0X
ExternalAppendOnlyUnsafeRowArray            10380 / 10451         25.3          39.6       1.7X

Spilling with 10000 rows:                Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
UnsafeExternalSorter                             6 /    7         25.3          39.5       1.0X
ExternalAppendOnlyUnsafeRowArray                 6 /    7         26.3          38.0       1.0X