[SPARK-5818][SQL] unable to use "add jar" in hql #5393
Conversation
These tests will run only when compiled with -Phive
|
Hi, Can some one verify this patch. Thanks |
There was a problem hiding this comment.
We can't make this change as I think it means that we will always build hive with the spark assembly, correct?
There was a problem hiding this comment.
No, dependency is added in hive profile only. So if assembly is built with -Phive option then hive dependency is added to repl, so that hive is available in class path for repl.
If assembly is built without -Phive option then dependency is not added and the testcase is ignored. Testcase also checks runtime if hiveContext class is available, if not testcase is ignored.
I have manually tested both the cases of building with -Phive and without hive. It will not impact assembly creation.
|
Can one of the admins verify this patch? |
|
Thanks for working on this. Can we close this issue until you have time to bring it up to date? |
3 participants
TranslatingClassLoader is used for Spark-shell, while current hive's add jar can work only with URLClassLoader.
So jar has to be directly added to spark driver's class loader or its parent loader, in case of spark-shell