Skip to content

[SPARK-28420][SQL] Support the INTERVAL type in date_part() #25981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MaxGekk wants to merge 43 commits into from
Closed

[SPARK-28420][SQL] Support the INTERVAL type in date_part() #25981

Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
b75a80f
Add Millennium
MaxGekk Sep 29, 2019
b233c90
Add Century
MaxGekk Sep 29, 2019
6f017a6
Add Decade
MaxGekk Sep 29, 2019
ac1c3a8
Add Year
MaxGekk Sep 29, 2019
7265508
Add Quarter
MaxGekk Sep 29, 2019
554df71
Move MONTHS_PER_QUARTER up
MaxGekk Sep 29, 2019
d0f89f4
Eliminate a warning
MaxGekk Sep 30, 2019
8174fd5
Extend with ExpectsInputTypes
MaxGekk Sep 30, 2019
6378b95
Remove blank lines
MaxGekk Sep 30, 2019
8e4ca7d
Add Month
MaxGekk Sep 30, 2019
8ed01c7
Week is not supported by PostgreSQL
MaxGekk Sep 30, 2019
f1eea12
Remove not-supported fields
MaxGekk Sep 30, 2019
561f789
Week of interval is not supported
MaxGekk Sep 30, 2019
0a671a9
Add Day
MaxGekk Sep 30, 2019
894a6c7
Add Hour
MaxGekk Sep 30, 2019
f86f4f5
Change Month type to ByteType
MaxGekk Sep 30, 2019
283fd99
Change Quarter type to ByteType
MaxGekk Sep 30, 2019
5e189ca
Put common code to IntervalPart
MaxGekk Sep 30, 2019
62c21b9
Run scalafmt
MaxGekk Sep 30, 2019
a4fbb5e
Add Minute
MaxGekk Sep 30, 2019
e1c9415
Add Second
MaxGekk Sep 30, 2019
7f4100f
Run scalafmt
MaxGekk Sep 30, 2019
f3cf7f0
Refactoring
MaxGekk Sep 30, 2019
9262f9d
Add a test for overflow
MaxGekk Sep 30, 2019
b9890ec
Add Milliseconds
MaxGekk Sep 30, 2019
bea3faf
Add Microseconds
MaxGekk Sep 30, 2019
77e0fb3
Add Epoch
MaxGekk Sep 30, 2019
58017a7
Support intervals by date_part
MaxGekk Sep 30, 2019
f5620b3
Update comments for DatePart
MaxGekk Sep 30, 2019
dcaf5b2
Regenerate results of date_part.sql
MaxGekk Sep 30, 2019
f202b15
Add tests for intervals to date_part.sql
MaxGekk Sep 30, 2019
dca29e5
Remove wrong test
MaxGekk Sep 30, 2019
f8a2385
Make Dongjoon and Scala style checker happy
MaxGekk Oct 1, 2019
7b1663e
Merge remote-tracking branch 'remotes/origin/master' into extract-fro…
MaxGekk Oct 8, 2019
8a494a2
Improve an example
MaxGekk Oct 8, 2019
f08531b
Precise epoch calculation from intervals
MaxGekk Oct 8, 2019
e8a61c8
Fix expected results in IntervalExpressionsSuite
MaxGekk Oct 8, 2019
f8a45b3
Revert "Precise epoch calculation from intervals"
MaxGekk Oct 12, 2019
a496d73
Precise calculation micros per month
MaxGekk Oct 12, 2019
47a0290
Revert "Precise calculation micros per month"
MaxGekk Oct 18, 2019
2099a91
Fix expected results in IntervalExpressionsSuite
MaxGekk Oct 18, 2019
d4375b5
Add the Extract prefix to all classes
MaxGekk Oct 18, 2019
5620472
Change indentation for extends to 2 spaces
MaxGekk Oct 18, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Eliminate a warning
  • Loading branch information
@MaxGekk
MaxGekk committed Sep 30, 2019
commit d0f89f41a6543659e533eba22f599e3f7c7dc0f5
2 changes: 2 additions & 0 deletions ...t/scala/org/apache/spark/sql/catalyst/expressions/interval/IntervalExpressionsSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@

package org.apache.spark.sql.catalyst.expressions.interval
Copy link
Member

@dongjoon-hyun dongjoon-hyun Oct 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto. Let's remove the interval directory and move this suite to the parent directory.

Copy link
Member Author

@MaxGekk MaxGekk Oct 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the interval related expressions to the separate package to not clash to date-time expressions. Though I renamed interval expressions in f3cf7f0 but having Millennium and Millenniums in the same name space looks confusing, doesn't it?

Copy link
Member

@dongjoon-hyun dongjoon-hyun Oct 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New changes look better to me.


import scala.language.implicitConversions

import org.apache.spark.SparkFunSuite
import org.apache.spark.sql.catalyst.expressions.{ExpressionEvalHelper, Literal}
import org.apache.spark.unsafe.types.CalendarInterval
Expand Down