Skip to content

[SPARK-24937][SQL] Datasource partition table should load empty static partitions #21883

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wangyum wants to merge 5 commits into from
Closed

[SPARK-24937][SQL] Datasource partition table should load empty static partitions #21883

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
90f99fc
Datasource partition table should load empty partitions.
wangyum Jul 26, 2018
d684d38
Fix test error.
wangyum Jul 26, 2018
b44e578
Add more test case
wangyum Jul 27, 2018
46bfcc6
Fix order
wangyum Jul 31, 2018
536346e
Add test case to test hive table
wangyum Aug 1, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix test error.
  • Loading branch information
@wangyum
wangyum committed Jul 26, 2018
commit d684d38c6731f84d568a8e42f6ff0455cf6ba700
4 changes: 2 additions & 2 deletions .../scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,8 +166,8 @@ case class InsertIntoHadoopFsRelationCommand(


// update metastore partition metadata
if (staticPartitions.nonEmpty) {
// Avoid empty partition can't loaded.
if (staticPartitions.nonEmpty && updatedPartitionPaths.isEmpty) {
// Avoid empty static partition can't loaded to datasource table.
refreshUpdatedPartitions(Set(PartitioningUtils.getPathFragment(staticPartitions)))
} else {
refreshUpdatedPartitions(updatedPartitionPaths)
Expand Down