Skip to content

[SPARK-31755][SQL][FOLLOWUP] Update date-time, CSV and JSON benchmark results #28613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MaxGekk wants to merge 2 commits into from
Closed

[SPARK-31755][SQL][FOLLOWUP] Update date-time, CSV and JSON benchmark results #28613

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3719110
Re-gen date-time, CSV and JSON benchmark results
MaxGekk May 22, 2020
117b3d0
Re-gen date-time, CSV and JSON benchmark results on JDK 8
MaxGekk May 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 44 additions & 44 deletions sql/core/benchmarks/CSVBenchmark-jdk11-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,66 +2,66 @@
Benchmark to measure CSV read/write performance
================================================================================================

Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.4
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Parsing quoted values: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
One quoted string 24907 29374 NaN 0.0 498130.5 1.0X
One quoted string 46568 46683 198 0.0 931358.6 1.0X

Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.4
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Wide rows with 1000 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Select 1000 columns 62811 63690 1416 0.0 62811.4 1.0X
Select 100 columns 23839 24064 230 0.0 23839.5 2.6X
Select one column 19936 20641 827 0.1 19936.4 3.2X
count() 4174 4380 206 0.2 4174.4 15.0X
Select 100 columns, one bad input field 41015 42380 1688 0.0 41015.4 1.5X
Select 100 columns, corrupt record field 46281 46338 93 0.0 46280.5 1.4X
Select 1000 columns 129836 130796 1404 0.0 129836.0 1.0X
Select 100 columns 40444 40679 261 0.0 40443.5 3.2X
Select one column 33429 33475 73 0.0 33428.6 3.9X
count() 7967 8047 73 0.1 7966.7 16.3X
Select 100 columns, one bad input field 90639 90832 266 0.0 90638.6 1.4X
Select 100 columns, corrupt record field 109023 109084 74 0.0 109023.3 1.2X

Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.4
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Count a dataset with 10 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Select 10 columns + count() 10810 10997 163 0.9 1081.0 1.0X
Select 1 column + count() 7608 7641 47 1.3 760.8 1.4X
count() 2415 2462 77 4.1 241.5 4.5X
Select 10 columns + count() 20685 20707 35 0.5 2068.5 1.0X
Select 1 column + count() 13096 13149 49 0.8 1309.6 1.6X
count() 3994 4001 7 2.5 399.4 5.2X

Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.4
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Write dates and timestamps: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Create a dataset of timestamps 874 914 37 11.4 87.4 1.0X
to_csv(timestamp) 7051 7223 250 1.4 705.1 0.1X
write timestamps to files 6712 6741 31 1.5 671.2 0.1X
Create a dataset of dates 909 945 35 11.0 90.9 1.0X
to_csv(date) 4222 4231 8 2.4 422.2 0.2X
write dates to files 3799 3813 14 2.6 379.9 0.2X
Create a dataset of timestamps 2169 2203 32 4.6 216.9 1.0X
to_csv(timestamp) 14401 14591 168 0.7 1440.1 0.2X
write timestamps to files 13209 13276 59 0.8 1320.9 0.2X
Create a dataset of dates 2231 2248 17 4.5 223.1 1.0X
to_csv(date) 10406 10473 68 1.0 1040.6 0.2X
write dates to files 7970 7976 9 1.3 797.0 0.3X

Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.4
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Read dates and timestamps: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
read timestamp text from files 1342 1364 35 7.5 134.2 1.0X
read timestamps from files 20300 20473 247 0.5 2030.0 0.1X
infer timestamps from files 40705 40744 54 0.2 4070.5 0.0X
read date text from files 1146 1151 6 8.7 114.6 1.2X
read date from files 12278 12408 117 0.8 1227.8 0.1X
infer date from files 12734 12872 220 0.8 1273.4 0.1X
timestamp strings 1467 1482 15 6.8 146.7 0.9X
parse timestamps from Dataset[String] 21708 22234 477 0.5 2170.8 0.1X
infer timestamps from Dataset[String] 42357 43253 922 0.2 4235.7 0.0X
date strings 1512 1532 18 6.6 151.2 0.9X
parse dates from Dataset[String] 13436 13470 33 0.7 1343.6 0.1X
from_csv(timestamp) 20390 20486 95 0.5 2039.0 0.1X
from_csv(date) 12592 12693 139 0.8 1259.2 0.1X
read timestamp text from files 2387 2391 6 4.2 238.7 1.0X
read timestamps from files 53503 53593 124 0.2 5350.3 0.0X
infer timestamps from files 107988 108668 647 0.1 10798.8 0.0X
read date text from files 2121 2133 12 4.7 212.1 1.1X
read date from files 29983 30039 48 0.3 2998.3 0.1X
infer date from files 30196 30436 218 0.3 3019.6 0.1X
timestamp strings 3098 3109 10 3.2 309.8 0.8X
parse timestamps from Dataset[String] 63331 63426 84 0.2 6333.1 0.0X
infer timestamps from Dataset[String] 124003 124463 490 0.1 12400.3 0.0X
date strings 3423 3429 11 2.9 342.3 0.7X
parse dates from Dataset[String] 34235 34314 76 0.3 3423.5 0.1X
from_csv(timestamp) 60829 61600 668 0.2 6082.9 0.0X
from_csv(date) 33047 33173 139 0.3 3304.7 0.1X

Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.4
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Filters pushdown: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
w/o filters 12535 12606 67 0.0 125348.8 1.0X
pushdown disabled 12611 12672 91 0.0 126112.9 1.0X
w/ filters 1093 1099 11 0.1 10928.3 11.5X
w/o filters 28752 28765 16 0.0 287516.5 1.0X
pushdown disabled 28856 28880 22 0.0 288556.3 1.0X
w/ filters 1714 1731 15 0.1 17137.3 16.8X


Loading