-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-35360][SQL] RepairTableCommand respects spark.sql.addPartitionInBatch.size too
#32489
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
MaxGekk
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need to change the config doc since it mentions the concrete command:
| "`AlterTableAddPartitionCommand` to add partitions into table. The smaller " + |
|
RepairTableCommand respect -> RepairTableCommand respects |
spark.sql.addPartitionInBatch.size toospark.sql.addPartitionInBatch.size too
Done |
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
|
+1, LGTM. GA passed. Merging to master. |
|
Test build #138325 has finished for PR 32489 at commit
|
What changes were proposed in this pull request?
RepairTableCommand respects
spark.sql.addPartitionInBatch.sizetooWhy are the changes needed?
Make RepairTableCommand add partition batch size configurable.
Does this PR introduce any user-facing change?
User can use
spark.sql.addPartitionInBatch.sizeto change batch size when repair table.How was this patch tested?
Not need