[SPARK-22676][FOLLOW-UP] fix code style for test. #21091
Conversation
|
Jenkins, test this please. |
|
Test build #89481 has finished for PR 21091 at commit
|
|
cc @cloud-fan |
|
|
||
| sql("DROP TABLE IF EXISTS table_with_partition") | ||
| sql("DROP TABLE IF EXISTS createAndInsertTest") | ||
| withSQLConf((SQLConf.HIVE_VERIFY_PARTITION_PATH.key -> "true")) { |
There was a problem hiding this comment.
we only need one pair of ()
| withTempView("testData") { | ||
| withTable("table_with_partition", "createAndInsertTest") { | ||
| withTempPath { tmpDir => | ||
| tmpDir.mkdir() |
There was a problem hiding this comment.
withTempDir, then you don't need to call mkdir here
|
|
||
| // test for the exist path | ||
| checkAnswer(sql("select key,value from table_with_partition"), | ||
| testData.union(testData).union(testData).union(testData).collect) |
There was a problem hiding this comment.
no need to call collect, checkAnswer can compare 2 dataframes
jinxing64
force-pushed
the
SPARK-22676-FOLLOW-UP
branch
from
59edf82 to
e9ff4d0
Compare
|
Test build #89543 has finished for PR 21091 at commit
|
|
Test build #89541 has finished for PR 21091 at commit
|
| } | ||
| } | ||
|
|
||
| test("Replace spark.sql.hive.verifyPartitionPath by spark.files.ignoreMissingFiles") { |
There was a problem hiding this comment.
These 2 tests are exactly same except the config, can we create a common method for it?
There was a problem hiding this comment.
I extracted a common method, check again.
jinxing64
force-pushed
the
SPARK-22676-FOLLOW-UP
branch
from
4838b79 to
e9ff4d0
Compare
|
Test build #89562 has finished for PR 21091 at commit
|
|
thanks, merging to master! |
|
Thanks for merging ! |
3 participants
What changes were proposed in this pull request?
This pr address comments in #19868 ;
Fix the code style for
org.apache.spark.sql.hive.QueryPartitionSuiteby using:withTempView,withTempDir,withTable...