[SPARK-34815][SQL] Update CSVBenchmark

[HyukjinKwon]

What changes were proposed in this pull request?

This PR updates CSVBenchmark especially we have a fix like #31858 that could potentially improve the performance.

Why are the changes needed?

To have the updated benchmark results.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manually ran the benchmark

[dongjoon-hyun]

Could you update Java 11 result together?

$ ls CSV*
CSVBenchmark-jdk11-results.txt
CSVBenchmark-results.txt

[HyukjinKwon]

Sure, running now 👍

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40904/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40904/

[SparkQA]

Test build #136322 has finished for PR 31917 at commit 750f92b.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[MaxGekk]

@HyukjinKwon Could you update PR's description and point out the environment in which you run the benchmark, please.

[HyukjinKwon]

I think the benchmark results include that.

[MaxGekk]

@HyukjinKwon The purpose is to give others enough info about the environment to get the same benchmark results. Do you really think that:

Java HotSpot(TM) 64-Bit Server VM 11.0.3+12-LTS on Mac OS X 10.15.7
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz

is enough? ok, how much memory should I have? 1MB RAM is enough?

[HyukjinKwon]

@MaxGekk If we care about that, it would be great if we include that in benchmark results.

[MaxGekk]

@HyukjinKwon I care of reproducible benchmark results. Currently, you don't provide enough info to reproduce the same. I would prefer to follow scientific approach, and have a chance to verify your results if it is needed.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40912/

[HyukjinKwon]

@MaxGekk, We should better have a way to do that, or at least document that we should do extra steps. All I read is:

spark/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVBenchmark.scala

Lines 30 to 40 in d65f534

	* Benchmark to measure CSV read/write performance.
	* To run this benchmark:
	* {{{
	* 1. without sbt:
	* bin/spark-submit --class <this class> --jars <spark core test jar>,
	* <spark catalyst test jar> <spark sql test jar>
	* 2. build/sbt "sql/test:runMain <this class>"
	* 3. generate result:
	* SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain <this class>"
	* Results will be written to "benchmarks/CSVBenchmark-results.txt".
	* }}}

If there are extra steps to do it, let's start another discussion and document it (FWIW I personally don't agree with having extra steps). It would be great if we have an automated script.

Until we have them, I don't think it's something required. I already see other envs were used in the past benchmark results.

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40912/

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40914/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40914/

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40915/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40915/

[HyukjinKwon]

I had an offline discussion with @MaxGekk.

I'm thinking about setting a GitHub Actions workflow like "Running tests in your forked repository using GitHub Actions" https://spark.apache.org/developer-tools.html, and we run the benchmark always in GA machines.

I guess the machine specifications are still not guaranteed to be same but would expect less variance compared to non-pinned env, and should be very easy for other people to run (just go to your fork, run a benchmark by UI, and download the benchmark results). I will try to take a look probably this week.

Meanwhile, I think we can just unblock this PR and go ahead.

[MaxGekk]

+1, LGTM, Merging this to master.

[HyukjinKwon]

Thank you @MaxGekk!

[HyukjinKwon]

I filed a JIRA for that: SPARK-34821

[wangyum]

@HyukjinKwon @MaxGekk We can use Hosting your own runners. This is an example:
https://github.com/wangyum/spark/blob/test-ci/.github/workflows/benchmark.yml#L11
https://github.com/wangyum/spark/runs/2164700670

[SparkQA]

Test build #136331 has finished for PR 31917 at commit 3575e48.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.