Skip to content

[MINOR][YARN] Add YARN-specific credential providers in debug logging message #21466

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
HyukjinKwon wants to merge 5 commits into from
Closed

[MINOR][YARN] Add YARN-specific credential providers in debug logging message #21466

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5a2b7c7
Add YARN-specific credential providers in debug logging message
HyukjinKwon May 31, 2018
3b6d45f
Address a comment
HyukjinKwon May 31, 2018
4a63f22
nit
HyukjinKwon May 31, 2018
046d433
nit
HyukjinKwon May 31, 2018
210ac14
credentialProviders.key.nonEmpty -> credentialProviders.nonEmpty
HyukjinKwon May 31, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ import org.apache.spark.internal.Logging
*
* Also, each HadoopDelegationTokenProvider is controlled by
* spark.security.credentials.{service}.enabled, and will not be loaded if this config is set to
* false. For example, Hive's delegation token provider [[HiveDelegationTokenProvider]] can be
* false. For example, Hive's delegation token provider [[HiveDelegationTokenProvider]] can be
* enabled/disabled by the configuration spark.security.credentials.hive.enabled.
*
* @param sparkConf Spark configuration
Expand All @@ -52,7 +52,7 @@ private[spark] class HadoopDelegationTokenManager(

// Maintain all the registered delegation token providers
private val delegationTokenProviders = getDelegationTokenProviders
logDebug(s"Using the following delegation token providers: " +
logDebug(s"Using the following builtin delegation token providers: " +
s"${delegationTokenProviders.keys.mkString(", ")}.")

/** Construct a [[HadoopDelegationTokenManager]] for the default Hadoop filesystem */
Expand Down
2 changes: 2 additions & 0 deletions ...c/main/scala/org/apache/spark/deploy/yarn/security/YARNHadoopDelegationTokenManager.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ private[yarn] class YARNHadoopDelegationTokenManager(

// public for testing
val credentialProviders = getCredentialProviders
logDebug(s"Using the following YARN-specific credential providers: " +
s"${credentialProviders.keys.mkString(", ")}.")
Copy link
Contributor

@jerryshao jerryshao May 31, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think usually this will be empty, we should only output this log when there is credential provider loaded in.

Copy link
Member Author

@HyukjinKwon HyukjinKwon May 31, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.


/**
* Writes delegation tokens to creds. Delegation tokens are fetched from all registered
Expand Down