[SPARK-31477][SQL] Dump codegen and compile time in BenchmarkQueryTest #28252
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gatorsmile
referenced
this pull request
Apr 18, 2020
…for generated java code ### What changes were proposed in this pull request? After my investigation, `SQLQueryTestSuite` spent a lot of time compiling the generated java code. Take `group-by.sql` as an example. At first, I added some debug log into `SQLQueryTestSuite`. Please reference https://github.com/beliefer/spark/blob/92b6af740c73e33007c21592023eb65635bb0f07/sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala#L402 The execution command is as follows: `build/sbt "~sql/test-only *SQLQueryTestSuite -- -z group-by.sql"` The output show below: ``` 00:56:06.192 WARN org.apache.spark.sql.SQLQueryTestSuite: group-by.sql using configs: spark.sql.codegen.wholeStage=true. run time: 20604 00:56:13.719 WARN org.apache.spark.sql.SQLQueryTestSuite: group-by.sql using configs: spark.sql.codegen.wholeStage=false,spark.sql.codegen.factoryMode=CODEGEN_ONLY. run time: 7526 00:56:18.786 WARN org.apache.spark.sql.SQLQueryTestSuite: group-by.sql using configs: spark.sql.codegen.wholeStage=false,spark.sql.codegen.factoryMode=NO_CODEGEN. run time: 5066 ``` According to the log, we know. Config | Run time(ms) -- | -- spark.sql.codegen.wholeStage=true | 20604 spark.sql.codegen.wholeStage=false,spark.sql.codegen.factoryMode=CODEGEN_ONLY | 7526 spark.sql.codegen.wholeStage=false,spark.sql.codegen.factoryMode=NO_CODEGEN | 5066 We should display the total compile time for generated java code. This PR will add the following to `SQLQueryTestSuite`'s output. ``` === Metrics of Whole Codegen === Total compile time: 80.564516529 seconds ``` Note: At first, I wanted to use `CodegenMetrics.METRIC_COMPILATION_TIME` to do this. After many experiments, I found that `CodegenMetrics.METRIC_COMPILATION_TIME` is only effective for a single test case, and cannot play a role in the whole life cycle of `SQLQueryTestSuite`. I checked the type of ` CodegenMetrics.METRIC_COMPILATION_TIME` is `Histogram` and the latter preserves 1028 elements.` Histogram` is a metric which calculates the distribution of a value. ### Why are the changes needed? Display the total compile time for generated java code. ### Does this PR introduce any user-facing change? 'No'. ### How was this patch tested? Jenkins test. Closes #28081 from beliefer/output-codegen-compile-time. Lead-authored-by: beliefer <[email protected]> Co-authored-by: gengjiaan <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
gatorsmile
force-pushed
the
testMastercode
branch
from
35e3803 to
2442370
Compare
Member
Author
maropu
approved these changes
Apr 18, 2020
|
Test build #121442 has finished for PR 28252 at commit
|
|
Test build #121443 has finished for PR 28252 at commit
|
Member
|
Retest this please. |
|
Test build #121445 has finished for PR 28252 at commit
|
Member
|
Thanks, all! Merged to master. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR is to dump the codegen and compilation time for benchmark query tests.
Why are the changes needed?
Measure the codegen and compilation time costs in TPC-DS queries
Does this PR introduce any user-facing change?
No
How was this patch tested?
Manual test in my local laptop: