[SPARK-36339][SQL][3.0] References to grouping that not part of aggregation should be replaced #33669
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…n should be replaced
Currently, references to grouping sets are reported as errors after aggregated expressions, e.g.
```
SELECT count(name) c, name
FROM VALUES ('Alice'), ('Bob') people(name)
GROUP BY name GROUPING SETS(name);
```
Error in query: expression 'people.`name`' is neither present in the group by, nor is it an aggregate function. Add to group by or wrap in first() (or first_value) if you don't care which value you get.;;
Fix the map anonymous function in the constructAggregateExprs function does not use underscores to avoid
No
Unit tests.
Closes apache#33574 from gaoyajun02/SPARK-36339.
Lead-authored-by: gaoyajun02 <[email protected]>
Co-authored-by: gaoyajun02 <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 888f8f0)
Contributor
|
ok to test |
Contributor
|
can we also add the test? |
|
Test build #142163 has started for PR 33669 at commit |
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
Contributor
Author
|
Sorry for the missing UT, added |
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
|
Test build #142186 has finished for PR 33669 at commit
|
cloud-fan
pushed a commit
that referenced
this pull request
Aug 9, 2021
…gation should be replaced
### What changes were proposed in this pull request?
Currently, references to grouping sets are reported as errors after aggregated expressions, e.g.
```
SELECT count(name) c, name
FROM VALUES ('Alice'), ('Bob') people(name)
GROUP BY name GROUPING SETS(name);
```
Error in query: expression 'people.`name`' is neither present in the group by, nor is it an aggregate function. Add to group by or wrap in first() (or first_value) if you don't care which value you get.;;
### Why are the changes needed?
Fix the map anonymous function in the constructAggregateExprs function does not use underscores to avoid
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Unit tests.
Closes #33669 from gaoyajun02/branch-3.0.
Authored-by: gaoyajun02 <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
cloud-fan
pushed a commit
that referenced
this pull request
Aug 9, 2021
…gation should be replaced
Currently, references to grouping sets are reported as errors after aggregated expressions, e.g.
```
SELECT count(name) c, name
FROM VALUES ('Alice'), ('Bob') people(name)
GROUP BY name GROUPING SETS(name);
```
Error in query: expression 'people.`name`' is neither present in the group by, nor is it an aggregate function. Add to group by or wrap in first() (or first_value) if you don't care which value you get.;;
Fix the map anonymous function in the constructAggregateExprs function does not use underscores to avoid
No
Unit tests.
Closes #33669 from gaoyajun02/branch-3.0.
Authored-by: gaoyajun02 <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit e10cc78)
Signed-off-by: Wenchen Fan <[email protected]>
Contributor
|
thanks, merging to 3.0/3.1! |
flyrain
pushed a commit
to flyrain/spark
that referenced
this pull request
Sep 21, 2021
…gation should be replaced
Currently, references to grouping sets are reported as errors after aggregated expressions, e.g.
```
SELECT count(name) c, name
FROM VALUES ('Alice'), ('Bob') people(name)
GROUP BY name GROUPING SETS(name);
```
Error in query: expression 'people.`name`' is neither present in the group by, nor is it an aggregate function. Add to group by or wrap in first() (or first_value) if you don't care which value you get.;;
Fix the map anonymous function in the constructAggregateExprs function does not use underscores to avoid
No
Unit tests.
Closes apache#33669 from gaoyajun02/branch-3.0.
Authored-by: gaoyajun02 <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit e10cc78)
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit e1a549c)
Signed-off-by: Dongjoon Hyun <[email protected]>
fishcus
pushed a commit
to fishcus/spark
that referenced
this pull request
Jan 12, 2022
…gation should be replaced
Currently, references to grouping sets are reported as errors after aggregated expressions, e.g.
```
SELECT count(name) c, name
FROM VALUES ('Alice'), ('Bob') people(name)
GROUP BY name GROUPING SETS(name);
```
Error in query: expression 'people.`name`' is neither present in the group by, nor is it an aggregate function. Add to group by or wrap in first() (or first_value) if you don't care which value you get.;;
Fix the map anonymous function in the constructAggregateExprs function does not use underscores to avoid
No
Unit tests.
Closes apache#33669 from gaoyajun02/branch-3.0.
Authored-by: gaoyajun02 <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit e10cc78)
Signed-off-by: Wenchen Fan <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Currently, references to grouping sets are reported as errors after aggregated expressions, e.g.
Error in query: expression 'people.
name' is neither present in the group by, nor is it an aggregate function. Add to group by or wrap in first() (or first_value) if you don't care which value you get.;;Why are the changes needed?
Fix the map anonymous function in the constructAggregateExprs function does not use underscores to avoid
Does this PR introduce any user-facing change?
No
How was this patch tested?
Unit tests.