Skip to content

[SPARK-32481][CORE][SQL] Support truncate table to move data to trash #29387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Udbhav30 wants to merge 7 commits into from
Closed

[SPARK-32481][CORE][SQL] Support truncate table to move data to trash #29387

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
58d957a
[SPARK-32481] Support truncate table to move data to trash
Udbhav30 Aug 8, 2020
81101d9
Handle review comments
Udbhav30 Aug 21, 2020
502dd08
add a warning msg
Udbhav30 Aug 21, 2020
97d2146
add tests and minor fixes
Udbhav30 Aug 22, 2020
a52e9a5
add tests and minor fixes
Udbhav30 Aug 22, 2020
8848b3f
typo fix
Udbhav30 Aug 22, 2020
a2df53b
Review comments fix
Udbhav30 Aug 24, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add a warning msg
  • Loading branch information
@Udbhav30
Udbhav30 committed Aug 24, 2020
commit 502dd08e99eaed2bc71b49cdfb98bdaf3a9cdec1
5 changes: 4 additions & 1 deletion core/src/main/scala/org/apache/spark/util/Utils.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,10 @@ private[spark] object Utils extends Logging {
hadoopConf: Configuration): Unit = {
if (isTrashEnabled && hadoopConf.getInt("fs.trash.interval", 0) > 0) {
logDebug(s"will move data ${partitionPath.toString} to trash")
Trash.moveToAppropriateTrash(fs, partitionPath, hadoopConf)
val isSuccess = Trash.moveToAppropriateTrash(fs, partitionPath, hadoopConf)
if (!isSuccess) {
logWarning(s"Failed to move data ${partitionPath.toString} to trash")
}
} else {
fs.delete(partitionPath, true)
}
Expand Down