-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-12344][CORE][YARN] Removal of SPARK_YARN_USER_ENV variable #12848
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Can one of the admins verify this patch? |
|
Hi @amits83 the bug is about removal of all env-based configuration, not just this single one. Have you looked at the original bug description, or any of the other env variables used in the code? There's a lot more work to achieve the goal of the bug. |
|
Hi @vanzin , yes I know that. This was my first commit so limited it to just one var at this time. :) I was looking at conf/spark-env.sh.template . Do you think thats a good starting point to find env variables listed in the template within the code and start removing them? |
|
@amits83 yes that file looks like a good place to start. Especially because a bunch of those have already been removed. If you don't plan to fix the whole bug, the usual approach is to open a sub-task for the specific thing you're fixing. In this case though, it would be better to avoid a bunch of really tiny PRs and take a more holistic look at the problem, since a lot of them will require the functionality that I described in the bug. |
|
@vanzin thanks for that. I dont intend to take a piecemeal approach. I will work on the other vars and file a PR for all of them together. Thanks again.
|
|
Hi @amits83, were you planning to also look at the other env variables? Otherwise should probably close this for the time being. |
|
Hi @vanzin yes I'm working on this. I intend to send in a PR soon. On Wed, Jun 8, 2016 at 4:31 PM, Marcelo Vanzin [email protected]
With Best Regards, |
|
@amits83 could you close this PR if you're not going to address the comments? thanks |
What changes were proposed in this pull request?
Removal of SPARK_YARN_USER_ENV env variable use in the code
How was this patch tested?
Ran ./dev/lint-scala to verify changes.