Skip to content

[SPARK-22856][SQL] Add wrappers for codegen output and nullability #20043

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
viirya wants to merge 14 commits into from
Closed

[SPARK-22856][SQL] Add wrappers for codegen output and nullability #20043

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
5ace8b8
Add wrappers for codegen output.
viirya Dec 21, 2017
d120750
Merge remote-tracking branch 'upstream/master' into SPARK-22856
viirya Dec 21, 2017
81c9b6e
Fix equality check with string.
viirya Dec 21, 2017
53926cc
Address comments.
viirya Dec 21, 2017
69422d4
Merge remote-tracking branch 'upstream/master' into SPARK-22856
viirya Dec 22, 2017
4384c84
Fix merging.
viirya Dec 22, 2017
d864aff
Merge remote-tracking branch 'upstream/master' into SPARK-22856
viirya Feb 21, 2018
8715d32
Add test.
viirya Feb 21, 2018
f59bb19
Use TrueLiteral/FalseLiteral. Add java type and access property to Ex…
viirya Feb 28, 2018
37ae9b0
Address comment.
viirya Mar 1, 2018
0841c4a
Address comment.
viirya Mar 1, 2018
e530f01
Remove ExprType.
viirya Mar 1, 2018
c8c70a9
Merge remote-tracking branch 'upstream/master' into SPARK-22856
viirya Mar 5, 2018
ac2e595
Merge remote-tracking branch 'upstream/master' into SPARK-22856
viirya Apr 4, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Address comments.
  • Loading branch information
@viirya
viirya committed Dec 21, 2017
commit 53926ccb0795c09a90ba70a5c7862ec1cb126391
2 changes: 1 addition & 1 deletion sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ abstract class Expression extends TreeNode[Expression] {
private def reduceCodeSize(ctx: CodegenContext, eval: ExprCode): Unit = {
// TODO: support whole stage codegen too
if (eval.code.trim.length > 1024 && ctx.INPUT_ROW != null && ctx.currentVars == null) {
val setIsNull = if ("false" != s"${eval.isNull}" && "true" != s"${eval.isNull}") {
val setIsNull = if (!eval.isNull.isInstanceOf[LiteralValue]) {
val globalIsNull = ctx.addMutableState(ctx.JAVA_BOOLEAN, "globalIsNull")
val localIsNull = eval.isNull
eval.isNull = GlobalValue(globalIsNull)
Expand Down
6 changes: 3 additions & 3 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/nullExpressions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ case class IsNull(child: Expression) extends UnaryExpression with Predicate {

override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
val eval = child.genCode(ctx)
val value = if ("true" == s"${eval.isNull}" || "false" == s"${eval.isNull}") {
val value = if (eval.isNull.isInstanceOf[LiteralValue]) {
LiteralValue(eval.isNull)
} else {
VariableValue(eval.isNull)
Expand Down Expand Up @@ -353,9 +353,9 @@ case class IsNotNull(child: Expression) extends UnaryExpression with Predicate {

override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
val eval = child.genCode(ctx)
val value = if ("true" == s"${eval.isNull}") {
val value = if (eval.isNull == LiteralValue("true")) {
LiteralValue("false")
} else if ("false" == s"${eval.isNull}") {
} else if (eval.isNull == LiteralValue("false")) {
LiteralValue("true")
} else {
StatementValue(s"(!(${eval.isNull}))")
Expand Down