[SPARK-11484][WebUi] Using proxyBase set by spark AM

[vundela]

Use the proxyBase set by the AM, if not found then use env. This is to fix the issue if somebody accidentally set APPLICATION_WEB_PROXY_BASE to wrong proxyBase

[vanzin]

ok to test

[SparkQA]

Test build #44966 has finished for PR 9448 at commit 1b272a4.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[vanzin]

I'd use the opportunity to clean this up a bit:

sys.props("spark.ui.proxyBase").orElse(sys.env("APPLICATION_WEB_PROXY_BASE")).getOrElse("")

[vanzin]

LGTM. My only concern is that someone might be using yarn-client mode with a different UI proxy than the RM; this change makes that impossible, since the AM always sets spark.ui.proxyBase. I find that case rather unlikely though.

[vundela]

Thanks for the review @vanzin, made the changes as you suggested.

[vanzin]

retest this please

[SparkQA]

Test build #44998 has finished for PR 9448 at commit b2a6838.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

I think this ultimately makes sense, letting Spark-specific stuff take precedence, though it's worth enumerating here or in the JIRA why this is connected to Oozie / uber mode, and why this is right-er than the current behavior.

[vundela]

Thanks for the review @srowen. Updated the JIRA to reflect the customer reported issue and why it is not related to Oozie/uber mode.

[vanzin]

Merging to master.