[SPARK-32999][SQL][2.4] Use Utils.getSimpleName to avoid hitting Malformed class name in TreeNode #29896
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… class name in TreeNode Use `Utils.getSimpleName` to avoid hitting `Malformed class name` error in `TreeNode`. On older JDK versions (e.g. JDK8u), nested Scala classes may trigger `java.lang.Class.getSimpleName` to throw an `java.lang.InternalError: Malformed class name` error. Similar to apache#29050, we should use Spark's `Utils.getSimpleName` utility function in place of `Class.getSimpleName` to avoid hitting the issue. Fixes a bug that throws an error when invoking `TreeNode.nodeName`, otherwise no changes. Added new unit test case in `TreeNodeSuite`. Note that the test case assumes the test code can trigger the expected error, otherwise it'll skip the test safely, for compatibility with newer JDKs. Manually tested on JDK8u and JDK11u and observed expected behavior: - JDK8u: the test case triggers the "Malformed class name" issue and the fix works; - JDK11u: the test case does not trigger the "Malformed class name" issue, and the test case is safely skipped. Closes apache#29875 from rednaxelafx/spark-32999-getsimplename. Authored-by: Kris Mok <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 9a155d4) Signed-off-by: Kris Mok <[email protected]>
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
maropu
approved these changes
Sep 28, 2020
HyukjinKwon
approved these changes
Sep 29, 2020
dongjoon-hyun
approved these changes
Sep 29, 2020
Member
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
+1, LGTM. Thank you, @rednaxelafx and all.
GitHub Action python UT failure is irrelevant one. I also verified this patch manually.
Merged to branch-2.4.
dongjoon-hyun
pushed a commit
that referenced
this pull request
Sep 29, 2020
…ormed class name in TreeNode # What changes were proposed in this pull request? Use `Utils.getSimpleName` to avoid hitting `Malformed class name` error in `TreeNode`. ### Why are the changes needed? On older JDK versions (e.g. JDK8u), nested Scala classes may trigger `java.lang.Class.getSimpleName` to throw an `java.lang.InternalError: Malformed class name` error. Similar to #29050, we should use Spark's `Utils.getSimpleName` utility function in place of `Class.getSimpleName` to avoid hitting the issue. ### Does this PR introduce _any_ user-facing change? Fixes a bug that throws an error when invoking `TreeNode.nodeName`, otherwise no changes. ### How was this patch tested? Added new unit test case in `TreeNodeSuite`. Note that the test case assumes the test code can trigger the expected error, otherwise it'll skip the test safely, for compatibility with newer JDKs. Manually tested on JDK8u and JDK11u and observed expected behavior: - JDK8u: the test case triggers the "Malformed class name" issue and the fix works; - JDK11u: the test case does not trigger the "Malformed class name" issue, and the test case is safely skipped. Closes #29896 from rednaxelafx/spark-32999-getsimplename-2.4. Authored-by: Kris Mok <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
|
Test build #129195 has finished for PR 29896 at commit
|
otterc
pushed a commit
to linkedin/spark
that referenced
this pull request
Mar 22, 2023
…ormed class name in TreeNode Use `Utils.getSimpleName` to avoid hitting `Malformed class name` error in `TreeNode`. On older JDK versions (e.g. JDK8u), nested Scala classes may trigger `java.lang.Class.getSimpleName` to throw an `java.lang.InternalError: Malformed class name` error. Similar to apache#29050, we should use Spark's `Utils.getSimpleName` utility function in place of `Class.getSimpleName` to avoid hitting the issue. Fixes a bug that throws an error when invoking `TreeNode.nodeName`, otherwise no changes. Added new unit test case in `TreeNodeSuite`. Note that the test case assumes the test code can trigger the expected error, otherwise it'll skip the test safely, for compatibility with newer JDKs. Manually tested on JDK8u and JDK11u and observed expected behavior: - JDK8u: the test case triggers the "Malformed class name" issue and the fix works; - JDK11u: the test case does not trigger the "Malformed class name" issue, and the test case is safely skipped. Closes apache#29896 from rednaxelafx/spark-32999-getsimplename-2.4. Authored-by: Kris Mok <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> RB=2667039 BUG=LIHADOOP-58072 G=spark-reviewers R=mmuralid A=mmuralid
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Use
Utils.getSimpleNameto avoid hittingMalformed class nameerror inTreeNode.Why are the changes needed?
On older JDK versions (e.g. JDK8u), nested Scala classes may trigger
java.lang.Class.getSimpleNameto throw anjava.lang.InternalError: Malformed class nameerror.Similar to #29050, we should use Spark's
Utils.getSimpleNameutility function in place ofClass.getSimpleNameto avoid hitting the issue.Does this PR introduce any user-facing change?
Fixes a bug that throws an error when invoking
TreeNode.nodeName, otherwise no changes.How was this patch tested?
Added new unit test case in
TreeNodeSuite. Note that the test case assumes the test code can trigger the expected error, otherwise it'll skip the test safely, for compatibility with newer JDKs.Manually tested on JDK8u and JDK11u and observed expected behavior: