Skip to content

[SPARK-23582][SQL] StaticInvoke should support interpreted execution #20753

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kiszk wants to merge 13 commits into from
Closed

[SPARK-23582][SQL] StaticInvoke should support interpreted execution #20753

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
9fea286
initial prototype
kiszk Mar 7, 2018
76c61ad
fix style error
kiszk Mar 7, 2018
80579c5
address review comments
kiszk Mar 8, 2018
d02e926
fix scala style error
kiszk Mar 8, 2018
b1f08c1
fix test failures
kiszk Mar 8, 2018
7d66bb9
move utility methods to ScalaReflection
kiszk Mar 26, 2018
c66e452
fix style error
kiszk Mar 26, 2018
1b3ea29
fix style error
kiszk Mar 26, 2018
7279557
address review comments
kiszk Mar 27, 2018
0530dcc
fix test failures
kiszk Mar 28, 2018
0c61e60
minimize changes
kiszk Apr 4, 2018
984cee7
fix build failure
kiszk Apr 4, 2018
7b30c30
fix scala style error
kiszk Apr 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix test failures
  • Loading branch information
@kiszk
kiszk committed Apr 4, 2018
commit 0530dcc2c32eb7ebcd86d66ede6f3fad9a6f797f
2 changes: 1 addition & 1 deletion sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,6 @@ case class StaticInvoke(
returnNullable: Boolean = true) extends InvokeLike {

val objectName = staticObject.getName.stripSuffix("$")
val argClasses = ScalaReflection.expressionJavaClasses(arguments)
val cls = if (staticObject.getName == objectName) {
staticObject
} else {
Expand All @@ -228,6 +227,7 @@ case class StaticInvoke(
override def nullable: Boolean = needNullCheck || returnNullable
override def children: Seq[Expression] = arguments

lazy val argClasses = ScalaReflection.expressionJavaClasses(arguments)
Copy link
Member Author

@kiszk kiszk Mar 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to make this lazy. This is because StaticInvoke is also called before dataType is resolved.
For example, test.org.apache.spark.sql.JavaDatasetSuite.test was failed without lazy.

@transient lazy val method = cls.getDeclaredMethod(functionName, argClasses : _*)

override def eval(input: InternalRow): Any = {
Expand Down