Skip to content

[SPARK-31797][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions #28534

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TJX2014 wants to merge 18 commits into from
Closed

[SPARK-31797][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions #28534

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
5e60ec5
SPARK-31710: TIMESTAMP_SECONDS, TIMESTAMP_MILLISECONDS and TIMESTAMP_…
TJX2014 May 19, 2020
27a162b
improve code quality according to advices
TJX2014 May 20, 2020
f879987
code style improved and remove bad asInstanceOf etc
TJX2014 May 20, 2020
4c17d49
From a readability perspective it may be better to put the base class…
TJX2014 May 20, 2020
3411935
new exception check when overflow occur when multiply long and more p…
TJX2014 May 20, 2020
17a33f5
long overflow in datetime.sql added and pass the local test
TJX2014 May 20, 2020
344e5e1
test overflow condition fix, just small unit for per test
TJX2014 May 20, 2020
407dcfc
fix unit test failed due to string value in ExpressionDescription
TJX2014 May 20, 2020
944c644
regenerate sql-expression-schema.md
TJX2014 May 20, 2020
642668c
minimize code gen by judge equal 1 and remove duplicate unit test
TJX2014 May 20, 2020
62bf8b5
correct style of usage content
TJX2014 May 21, 2020
7da94b5
short name to MillisToTimestamp and MicrosToTimestamp
TJX2014 May 21, 2020
3704fee
correct short name in sql-expression-schema.md
TJX2014 May 21, 2020
f5b0a5a
function rename timestamp_microseconds to timestamp_micros and timest…
TJX2014 May 21, 2020
f4f7c35
disable cast string to long in number to timestamp
TJX2014 May 21, 2020
3c949ac
regenerate sql-expression-schema.md, UT added in DateExpressionsSuite
TJX2014 May 22, 2020
fdfd75c
UT style improve
TJX2014 May 22, 2020
f0cc631
UT style improve
TJX2014 May 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
short name to MillisToTimestamp and MicrosToTimestamp
  • Loading branch information
@TJX2014
TJX2014 committed May 22, 2020
commit 7da94b52a73e365d5478467d8f1e83a59d0ddabb
4 changes: 2 additions & 2 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -425,8 +425,8 @@ object FunctionRegistry {
expression[DatePart]("date_part"),
expression[Extract]("extract"),
expression[SecondsToTimestamp]("timestamp_seconds"),
expression[MilliSecondsToTimestamp]("timestamp_milliseconds"),
expression[MicroSecondsToTimestamp]("timestamp_microseconds"),
expression[MillisToTimestamp]("timestamp_milliseconds"),
Copy link
Contributor

@cloud-fan cloud-fan May 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timestamp_millisecond -> timestamp_millis

Copy link
Member

@HyukjinKwon HyukjinKwon May 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also don't forget to update the PR title and description.

Copy link
Contributor Author

@TJX2014 TJX2014 May 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

expression[MicrosToTimestamp]("timestamp_microseconds"),
Copy link
Contributor

@cloud-fan cloud-fan May 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto


// collection functions
expression[CreateArray]("array"),
Expand Down
4 changes: 2 additions & 2 deletions ...talyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -451,7 +451,7 @@ case class SecondsToTimestamp(child: Expression)
group = "datetime_funcs",
since = "3.1.0")
// scalastyle:on line.size.limit
case class MilliSecondsToTimestamp(child: Expression)
case class MillisToTimestamp(child: Expression)
extends NumberToTimestampBase {

override def upScaleFactor: Long = MICROS_PER_MILLIS
Expand All @@ -470,7 +470,7 @@ case class MilliSecondsToTimestamp(child: Expression)
group = "datetime_funcs",
since = "3.1.0")
// scalastyle:on line.size.limit
case class MicroSecondsToTimestamp(child: Expression)
case class MicrosToTimestamp(child: Expression)
extends NumberToTimestampBase {

override def upScaleFactor: Long = 1L
Expand Down