Skip to content

[SPARK-36039][K8S] Fix executor pod hadoop conf mount #33257

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cutiechi wants to merge 4 commits into from
Closed

[SPARK-36039][K8S] Fix executor pod hadoop conf mount #33257

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ab865a7
Add hadoop conf executor feature step to kubernetes executor builder
cutiechi Jul 8, 2021
61c9372
Add hadoop conf executor feature step to kubernetes executor builder …
cutiechi Jul 19, 2021
2a91196
Fix executor pod hadoop conf config map create
cutiechi Jul 21, 2021
daee42b
Fix executor pod create or replace NPE
cutiechi Jul 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Fix executor pod create or replace NPE
  • Loading branch information
@cutiechi
cutiechi committed Jul 28, 2021
commit daee42b0ab0864edb5222feeb9fd8d1b60888d50
4 changes: 3 additions & 1 deletion ...es/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -389,7 +389,9 @@ private[spark] class ExecutorPodsAllocator(
s"StorageClass ${pvc.getSpec.getStorageClassName}")
kubernetesClient.persistentVolumeClaims().create(pvc)
}
kubernetesClient.resourceList(resources: _*).createOrReplace()
if (resources.nonEmpty) {
kubernetesClient.resourceList(resources: _*).createOrReplace()
}
newlyCreatedExecutors(newExecutorId) = (resourceProfileId, clock.getTimeMillis())
logDebug(s"Requested executor with id $newExecutorId from Kubernetes.")
} catch {
Expand Down