Skip to content

[SPARK-18016][SQL] Code Generation: Constant Pool Limit - reduce entries for mutable state #19811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kiszk wants to merge 26 commits into from
Closed

[SPARK-18016][SQL] Code Generation: Constant Pool Limit - reduce entries for mutable state #19811

Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
d441794
update API of addMutableState
kiszk Nov 24, 2017
870d106
eliminate initialization with default value
kiszk Nov 25, 2017
24d7087
allocate a global Java array to store a lot of mutable state in a class
kiszk Nov 26, 2017
3eb5842
fix scala style error
kiszk Nov 26, 2017
074d711
fix test failure of ExpressionEncoderSuite.NestedArray
kiszk Nov 27, 2017
eafa3f8
rebase with master
kiszk Nov 30, 2017
90d15f3
address review comments
kiszk Nov 30, 2017
c456c07
add useFreshname parameter to addMutableState
kiszk Nov 30, 2017
9ca5ab3
rebase with master
kiszk Nov 30, 2017
5b36c61
fix test failures
kiszk Dec 1, 2017
fd51d75
drop to creat a loop for initialization
kiszk Dec 7, 2017
effe918
fix test failure
kiszk Dec 8, 2017
9df109c
update comments
kiszk Dec 8, 2017
634d494
address review comment
kiszk Dec 10, 2017
d3438fd
address review comment
kiszk Dec 12, 2017
f4f3754
address review comment
kiszk Dec 12, 2017
f1e1fca
address review comments except test case
kiszk Dec 13, 2017
0937ef2
rebase with master
kiszk Dec 13, 2017
4bfcc1a
Do not use compaction as possible for frequently-accessed variable
kiszk Dec 13, 2017
49119a9
exclude mutable state from argument list for ExpressionCodegn
kiszk Dec 13, 2017
24f49c5
fix test failures
kiszk Dec 14, 2017
15e967e
address review comments
kiszk Dec 14, 2017
d6c1a97
address review comments
kiszk Dec 14, 2017
a9d40e9
address review comments
kiszk Dec 14, 2017
31914c0
address review comments
kiszk Dec 15, 2017
0e45c19
address review comments
kiszk Dec 19, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
rebase with master
  • Loading branch information
@kiszk
kiszk committed Dec 14, 2017
commit 0937ef20a57a7b0648bd3145c383d9645c67de2c
5 changes: 2 additions & 3 deletions ...lyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1122,9 +1122,8 @@ class CodegenContext {
// 2. Less code.
// Currently, we will do this for all non-leaf only expression trees (i.e. expr trees with
// at least two nodes) as the cost of doing it is expected to be low.
addMutableState(JAVA_BOOLEAN, isNull, s"$isNull = false;")
addMutableState(javaType(expr.dataType), value,
s"$value = ${defaultValue(expr.dataType)};")
addMutableState(JAVA_BOOLEAN, isNull, inline = true, useFreshName = false)
addMutableState(javaType(expr.dataType), value, inline = true, useFreshName = false)

subexprFunctions += s"${addNewFunction(fnName, fn)}($INPUT_ROW);"
val state = SubExprEliminationState(isNull, value)
Expand Down