Skip to content

[SPARK-48712][SQL] Perf Improvement for encode with empty values or UTF-8 charset #47096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yaooqinn wants to merge 7 commits into from
Closed

[SPARK-48712][SQL] Perf Improvement for encode with empty values or UTF-8 charset #47096

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8d36c1e
[SPARK-48712][SQL] Perf Improvement for encode with empty value or UT…
yaooqinn Jun 25, 2024
4d6c49d
add more case
yaooqinn Jun 26, 2024
64dccfe
add tests
yaooqinn Jun 26, 2024
8903ab6
update golden files for 17
yaooqinn Jun 26, 2024
7db3f31
update golden files for 21
yaooqinn Jun 26, 2024
5c4b5d0
add more tests
yaooqinn Jun 26, 2024
9a8c302
Merge branch 'master' into SPARK-48712
yaooqinn Jun 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update golden files for 17
  • Loading branch information
@yaooqinn
yaooqinn committed Jun 26, 2024
commit 8903ab644e27a0e17476b4f846de7c627fbca87a
10 changes: 5 additions & 5 deletions sql/core/benchmarks/EncodeBenchmark-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
OpenJDK 64-Bit Server VM 17.0.10+0 on Mac OS X 14.5
Apple M2 Max
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
encode: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
UTF-32 25263 25441 252 0.4 2526.3 1.0X
UTF-16 26975 27019 62 0.4 2697.5 0.9X
UTF-8 1886 1904 26 5.3 188.6 13.4X
UTF-32 31107 31205 138 0.3 3110.7 1.0X
UTF-16 47904 47934 43 0.2 4790.4 0.6X
UTF-8 2957 2978 30 3.4 295.7 10.5X