Skip to content

[SPARK-29786][SQL] Fix MetaException when dropping a partition not exists on HDFS #26422

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Deegue wants to merge 14 commits into from
Closed

[SPARK-29786][SQL] Fix MetaException when dropping a partition not exists on HDFS #26422

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
90233e4
Merge pull request #1 from apache/master
Deegue Oct 22, 2019
7a246d5
Merge pull request #2 from apache/master
Deegue Oct 28, 2019
783211f
Merge pull request #3 from apache/master
Deegue Nov 5, 2019
355496e
Fix drop partition MetaException:File does not exist.
Deegue Nov 7, 2019
74d5984
Add notes.
Deegue Nov 14, 2019
84b1da8
1584531350
Deegue Mar 18, 2020
248b6cf
UT failed #119927, getDataLocation->getPartitionPath
Deegue Mar 20, 2020
c20c390
UT failed #120076, add HIDDEN_FILES_PATH_FILTER
Deegue Mar 20, 2020
9d75854
optimize
Deegue Mar 20, 2020
5a5ff74
UT failed #120087. Hive-0.13 doesn't have `Partition.getPartitionPath…
Deegue Mar 20, 2020
f4b3793
UT failed #120093, purge and ignoreIfNotExists should be false.
Deegue Mar 20, 2020
4d1c35e
isEmptyPath->isExistPath
Deegue Mar 27, 2020
7cf7c6b
remove redundant code
Deegue Mar 27, 2020
f8b96de
restore '!'
Deegue May 29, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
UT failed #120093, purge and ignoreIfNotExists should be false.
  • Loading branch information
@Deegue
Deegue committed Mar 20, 2020
commit f4b3793afebee82a89c6ffa9f13c0187192496a7
4 changes: 2 additions & 2 deletions sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -547,8 +547,8 @@ class VersionsSuite extends SparkFunSuite with Logging {
fs.delete(partPath, true)
}
}
client.dropPartitions("default", "src_part", Seq(spec), ignoreIfNotExists = true,
purge = true, retainData = false)
client.dropPartitions("default", "src_part", Seq(spec), ignoreIfNotExists = false,
purge = false, retainData = false)
assert(client.getPartitionOption("default", "src_part", spec).isEmpty)
}

Expand Down