Skip to content

[Spark-21842][Mesos] Support Kerberos ticket renewal and creation in Mesos #19272

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ArtRand wants to merge 23 commits into from
Closed

[Spark-21842][Mesos] Support Kerberos ticket renewal and creation in Mesos #19272

Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
44a6098
basic implementation of MesosCredentialRenewer
Sep 19, 2017
781c5a7
pull upstream
Sep 19, 2017
18d2c6c
working through comments
Sep 20, 2017
43ab547
cleanup
Sep 20, 2017
f136eaa
remove unused imports
Sep 20, 2017
488f72a
remove debugging code
Sep 20, 2017
f5925fd
use 75 percent of renewal time for the first renewal also
Sep 26, 2017
3f22efe
addressed comments, still pending testing
Oct 16, 2017
e522150
final comments
Oct 18, 2017
837157d
add comments
Oct 20, 2017
c95f80b
remove extra thread in MesosCredentialRenewer
Oct 21, 2017
e8bbc9e
refactor credential renewer, addressed comments
Oct 30, 2017
1596d80
merge upstream
Oct 30, 2017
864ab7e
resolve import problem
Oct 30, 2017
7e0590a
cleanup
Oct 31, 2017
f93c551
more cleanup
Oct 31, 2017
b2fbcf2
move hadoop delegation token management into Mesos renewer
Nov 6, 2017
4558cea
remove unused imports
Nov 6, 2017
5f254e5
no more conditionals, start renewer automatically
Nov 8, 2017
8df7e37
addressed comments
Nov 10, 2017
45b46ed
quick fixup, logging was in the wrong place
Nov 10, 2017
18d77ff
only renew tokens when using keytab
Nov 14, 2017
049e4b5
update tokens, make SparkHadoopUtil methods private
Nov 15, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
quick fixup, logging was in the wrong place
  • Loading branch information
ArtRand committed Nov 10, 2017
commit 45b46ed6768ea50ddf23063b2a925c2a4794acc7
2 changes: 1 addition & 1 deletion core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ private[spark] class CoarseGrainedExecutorBackend(
}.start()

case UpdateDelegationTokens(tokenBytes) =>
logInfo(s"Received tokens of ${tokenBytes.length} bytes")
SparkHadoopUtil.get.addDelegationTokens(tokenBytes, env.conf)
Copy link
Contributor

@vanzin vanzin Nov 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a logInfo saying the tokens are being updated? This has always been helpful when debugging issues with this feature on YARN.

}

Expand Down Expand Up @@ -223,7 +224,6 @@ private[spark] object CoarseGrainedExecutorBackend extends Logging {
}

cfg.hadoopDelegationCreds.foreach { tokens =>
logInfo(s"Received tokens of ${tokens.length} bytes")
SparkHadoopUtil.get.addDelegationTokens(tokens, driverConf)
}

Expand Down