Skip to content

[SPARK-29515][Core] MapStatuses SerDeser Benchmark #26169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dbtsai wants to merge 5 commits into from
Closed

[SPARK-29515][Core] MapStatuses SerDeser Benchmark #26169

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8aeb1e2
MapStatusesSerDeserBenchmark
dbtsai Oct 18, 2019
91e2e7a
Add JDK11 result
dongjoon-hyun Oct 18, 2019
c3b1a0c
Add JDK8 result
dongjoon-hyun Oct 18, 2019
fbf9229
Merge pull request #6 from dongjoon-hyun/PR-26169
dbtsai Oct 18, 2019
8aa05c6
address feedback
dbtsai Oct 18, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add JDK8 result
  • Loading branch information
@dongjoon-hyun
dongjoon-hyun committed Oct 18, 2019
commit c3b1a0cff4617577f9aa948511c3879de5d093bf
66 changes: 66 additions & 0 deletions core/benchmarks/MapStatusesSerDeserBenchmark-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
OpenJDK 64-Bit Server VM 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
200000 MapOutputs, 10 blocks w/ broadcast: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Serialization 625 639 9 0.3 3127.2 1.0X
Deserialization 875 931 49 0.2 4376.2 0.7X

Compressed Serialized MapStatus sizes: 393 bytes
Compressed Serialized Broadcast MapStatus sizes: 3 MB


OpenJDK 64-Bit Server VM 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
200000 MapOutputs, 10 blocks w/o broadcast: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Serialization 604 640 71 0.3 3018.4 1.0X
Deserialization 889 903 17 0.2 4443.8 0.7X

Compressed Serialized MapStatus sizes: 3 MB
Compressed Serialized Broadcast MapStatus sizes: 0 bytes


OpenJDK 64-Bit Server VM 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
200000 MapOutputs, 100 blocks w/ broadcast: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Serialization 1879 1880 2 0.1 9394.9 1.0X
Deserialization 1147 1150 5 0.2 5733.8 1.6X

Compressed Serialized MapStatus sizes: 411 bytes
Compressed Serialized Broadcast MapStatus sizes: 15 MB


OpenJDK 64-Bit Server VM 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
200000 MapOutputs, 100 blocks w/o broadcast: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Serialization 1825 1826 1 0.1 9123.3 1.0X
Deserialization 1147 1281 189 0.2 5735.7 1.6X

Compressed Serialized MapStatus sizes: 15 MB
Compressed Serialized Broadcast MapStatus sizes: 0 bytes


OpenJDK 64-Bit Server VM 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
200000 MapOutputs, 1000 blocks w/ broadcast: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Serialization 12327 12518 270 0.0 61634.3 1.0X
Deserialization 3120 3133 18 0.1 15600.8 4.0X

Compressed Serialized MapStatus sizes: 544 bytes
Compressed Serialized Broadcast MapStatus sizes: 131 MB


OpenJDK 64-Bit Server VM 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
200000 MapOutputs, 1000 blocks w/o broadcast: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Serialization 11928 11986 82 0.0 59642.2 1.0X
Deserialization 3137 3138 2 0.1 15683.3 3.8X

Compressed Serialized MapStatus sizes: 131 MB
Compressed Serialized Broadcast MapStatus sizes: 0 bytes