Skip to content

[SPARK-29364][SQL] Return an interval from date subtract according to SQL standard #26112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MaxGekk wants to merge 14 commits into from
Closed

[SPARK-29364][SQL] Return an interval from date subtract according to SQL standard #26112

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
18bc09b
Add SubtractDates
MaxGekk Oct 14, 2019
141d3da
Add tests for SubtractDates
MaxGekk Oct 14, 2019
dc37bc7
Use SubtractDates for dates subtract
MaxGekk Oct 14, 2019
e82f027
Update the SQL migration guide
MaxGekk Oct 14, 2019
e74f81b
Regen date.sql.out
MaxGekk Oct 14, 2019
4a8173b
Regen datetime.sql.out
MaxGekk Oct 14, 2019
27de6b8
Add config spark.sql.legacy.datesSubtraction.enabled
MaxGekk Oct 14, 2019
fbd2723
TimestampDiff -> SubtractTimestamps
MaxGekk Oct 14, 2019
199adff
Update config description regarding datediff
MaxGekk Oct 14, 2019
8b49c9b
Regen datetime.sql.out
MaxGekk Oct 15, 2019
85f17f7
Revert "Update config description regarding datediff"
MaxGekk Oct 15, 2019
228673d
Revert "Add config spark.sql.legacy.datesSubtraction.enabled"
MaxGekk Oct 15, 2019
f232b64
Revert "Update the SQL migration guide"
MaxGekk Oct 15, 2019
c6ec211
Use DateDiff in the PostgreSQL dialect
MaxGekk Oct 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Regen date.sql.out
  • Loading branch information
@MaxGekk
MaxGekk committed Oct 14, 2019
commit e74f81b19f69608e93fa2cfa23f25d940a72c58b
88 changes: 44 additions & 44 deletions sql/core/src/test/resources/sql-tests/results/postgreSQL/date.sql.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -482,93 +482,93 @@ SELECT date '5874898-01-01'
-- !query 46
SELECT f1 - date '2000-01-01' AS `Days From 2K` FROM DATE_TBL
-- !query 46 schema
struct<Days From 2K:int>
struct<Days From 2K:interval>
-- !query 46 output
-1035
-1036
-1037
-1400
-1401
-1402
-1403
-15542
-15607
13977
14343
14710
91
92
93
interval -2 years -10 months
interval -2 years -10 months -1 days
interval -2 years -9 months -4 weeks -2 days
interval -3 years -10 months
interval -3 years -10 months -1 days
interval -3 years -10 months -2 days
interval -3 years -9 months -4 weeks -2 days
interval -42 years -6 months -2 weeks -4 days
interval -42 years -8 months -3 weeks -1 days
interval 3 months
interval 3 months 1 days
interval 3 months 2 days
interval 38 years 3 months 1 weeks
interval 39 years 3 months 1 weeks 1 days
interval 40 years 3 months 1 weeks 2 days

Copy link
Member

@wangyum wangyum Oct 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These results are inconsistent with PostgreSQL:
https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/expected/date.out#L845-L882


-- !query 47
SELECT f1 - date 'epoch' AS `Days From Epoch` FROM DATE_TBL
-- !query 47 schema
struct<Days From Epoch:int>
struct<Days From Epoch:interval>
-- !query 47 output
-4585
-4650
11048
11049
11050
24934
25300
25667
9554
9555
9556
9557
9920
9921
9922
interval -12 years -6 months -2 weeks -4 days
interval -12 years -8 months -3 weeks -1 days
interval 26 years 1 months 3 weeks 6 days
interval 26 years 1 months 4 weeks
interval 26 years 2 months
interval 26 years 2 months 1 days
interval 27 years 1 months 3 weeks 6 days
interval 27 years 2 months
interval 27 years 2 months 1 days
interval 30 years 3 months
interval 30 years 3 months 1 days
interval 30 years 3 months 2 days
interval 68 years 3 months 1 weeks
interval 69 years 3 months 1 weeks 1 days
interval 70 years 3 months 1 weeks 2 days


-- !query 48
SELECT date 'yesterday' - date 'today' AS `One day`
-- !query 48 schema
struct<One day:int>
struct<One day:interval>
-- !query 48 output
-1
interval -1 days


-- !query 49
SELECT date 'today' - date 'tomorrow' AS `One day`
-- !query 49 schema
struct<One day:int>
struct<One day:interval>
-- !query 49 output
-1
interval -1 days


-- !query 50
SELECT date 'yesterday' - date 'tomorrow' AS `Two days`
-- !query 50 schema
struct<Two days:int>
struct<Two days:interval>
-- !query 50 output
-2
interval -2 days


-- !query 51
SELECT date 'tomorrow' - date 'today' AS `One day`
-- !query 51 schema
struct<One day:int>
struct<One day:interval>
-- !query 51 output
1
interval 1 days


-- !query 52
SELECT date 'today' - date 'yesterday' AS `One day`
-- !query 52 schema
struct<One day:int>
struct<One day:interval>
-- !query 52 output
1
interval 1 days


-- !query 53
SELECT date 'tomorrow' - date 'yesterday' AS `Two days`
-- !query 53 schema
struct<Two days:int>
struct<Two days:interval>
-- !query 53 output
2
interval 2 days


-- !query 54
Expand Down