Skip to content

[SPARK-42551][SQL] Support more subexpression elimination cases #41119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wankunde wants to merge 10 commits into from
Closed

[SPARK-42551][SQL] Support more subexpression elimination cases #41119

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
ffe4ca8
[SPARK-42551][SQL] Support more subexpression elimination cases
wankunde Feb 24, 2023
db77ea2
Support whole stage subexpressions elimination
wankunde Apr 8, 2023
e5ee05e
Bug fix for whole stage subexpression elimination
wankunde Apr 13, 2023
f152b2a
Bug fix
wankunde Apr 13, 2023
9a7cdc6
Do not support CSE in produce method
wankunde Apr 14, 2023
076792f
Clear stale commonExpressions when reoptimize plan
wankunde Apr 14, 2023
f524ef5
Remove whole-stage expression elimination
wankunde May 9, 2023
670446a
Merge code into origin CSE
wankunde May 10, 2023
fbfa00b
bug fix
wankunde May 10, 2023
06238be
Consider common expression local input variables.
wankunde May 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
bug fix
  • Loading branch information
@wankunde
wankunde committed May 10, 2023
commit fbfa00b6e7fca682030a448c2d64e03cff595369
2 changes: 1 addition & 1 deletion ...lyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1307,7 +1307,7 @@ class CodegenContext extends Logging {
}

def genReusedCode(stats: ExpressionStats, eval: ExprCode): ExprCode = {
val (inputVars, _) = getLocalInputVariableValues(this, stats.expr)
val (inputVars, _) = getLocalInputVariableValues(this, stats.expr, subExprEliminationExprs)
val (initialized, isNull, value) = (stats.initialized.get, stats.isNull.get, stats.value.get)
val validParamLength = isValidParamLength(calculateParamLengthFromExprValues(inputVars))
if(!stats.addedFunction && validParamLength) {
Expand Down