Skip to content

[SPARK-28067][SQL] Fix incorrect results for decimal aggregate sum by returning null on decimal overflow #27627

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
skambha wants to merge 15 commits into from
Closed

[SPARK-28067][SQL] Fix incorrect results for decimal aggregate sum by returning null on decimal overflow #27627

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
edb32b9
Fix the incorrect results for sum for decimal overflow, support repor…
skambha Feb 13, 2020
8fe9aa4
Add the jira testcase
skambha Feb 13, 2020
1adc512
Change formatting
skambha Feb 13, 2020
d90c790
hasoverflow changes
skambha Feb 18, 2020
136e6dc
Fix formatting
skambha Mar 3, 2020
6979e8d
Add a new implementation for sum for dec type to handle overflow, Add…
skambha Apr 8, 2020
4119e02
Put back the decimal handling changes into Sum like how we started with
skambha Apr 9, 2020
cc2fec0
Use a new flag isEmptyOrNulls to handle the scenario with all nulls a…
skambha Apr 27, 2020
23739c9
Rebase
skambha Apr 27, 2020
fa45378
rebase
skambha Apr 28, 2020
fbd80a6
Add new test and also make the agg buffer changes only for decimal type
skambha May 15, 2020
de2d68f
simplify
cloud-fan May 21, 2020
8339e28
Fix the test failure
skambha May 29, 2020
59a00c4
Cleanup comments
skambha May 29, 2020
7795888
cleanup code comments
skambha Jun 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
rebase
  • Loading branch information
@skambha
skambha committed Apr 28, 2020
commit fa4537821337c45b3db26d6b4042a7103d044eac
4 changes: 2 additions & 2 deletions sql/core/src/test/scala/org/apache/spark/sql/ExplainSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -379,12 +379,12 @@ class ExplainSuiteAE extends ExplainSuiteHelper with EnableAdaptiveExecutionSuit
|""".stripMargin,
s"""
|(11) ShuffleQueryStage
|Output [5]: [k#x, count#xL, sum#xL, sum#x, count#xL]
|Output [6]: [k#x, count#xL, sum#xL, isEmptyOrNulls#x, sum#x, count#xL]
|Arguments: 1
|""".stripMargin,
s"""
|(12) CustomShuffleReader
|Input [5]: [k#x, count#xL, sum#xL, sum#x, count#xL]
|Input [6]: [k#x, count#xL, sum#xL, isEmptyOrNulls#x, sum#x, count#xL]
|Arguments: coalesced
|""".stripMargin,
s"""
Expand Down