Skip to content

[SPARK-21687][SQL] Spark SQL should set createTime for Hive partition #18900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
debugger87 wants to merge 13 commits into from
Closed

[SPARK-21687][SQL] Spark SQL should set createTime for Hive partition #18900

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
71a660a
[SPARK-21687][SQL] Spark SQL should set createTime for Hive partition
debugger87 Aug 10, 2017
f668ce8
added createTime and lastAccessTime into CatalogTablePartition
debugger87 Aug 11, 2017
2fb1dda
minor tweak
debugger87 Aug 11, 2017
c833ce7
fix type missmatch
debugger87 Aug 11, 2017
bf2a105
added createTime and lastAccessTime into partition map for display
debugger87 Aug 11, 2017
478e205
Merge branch 'master' into fix/set-create-time-for-hive-partition
debugger87 Jun 7, 2018
a00e943
bug fix
debugger87 Jun 7, 2018
e3a0cc4
added some comments
debugger87 Jun 7, 2018
b0846c3
fix missing comma
debugger87 Jun 7, 2018
0390e88
ignore createTime and lastAccessTime in catalogPartitionsEqual
debugger87 Jun 22, 2018
c843ef1
fix sql output
debugger87 Jun 22, 2018
715c7cc
regenerate output files for SQLQueryTestSuite after added Created Tim…
debugger87 Jun 27, 2018
18c85b6
fix comment
debugger87 Jun 27, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
added some comments
  • Loading branch information
@debugger87
debugger87 authored Jun 7, 2018
commit e3a0cc43b10828b8111f7cd9523391cd3a2fdb6f
2 changes: 2 additions & 0 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@ object CatalogStorageFormat {
* @param spec partition spec values indexed by column name
* @param storage storage format of the partition
* @param parameters some parameters for the partition
* @param createTime creation time of the partition
Copy link
Contributor

@cloud-fan cloud-fan Jun 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's mention the time unit, i.e. in milliseconds.

Copy link
Contributor Author

@debugger87 debugger87 Jun 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, it's the same as CatalogTable, in milliseconds. I fill fix this comment.

* @param lastAccessTime last access time
* @param stats optional statistics (number of rows, total size, etc.)
*/
case class CatalogTablePartition(
Expand Down