[SPARK-36462][K8S] Add the ability to selectively disable watching or polling#36433

Closed

holdenk wants to merge 11 commits intoapache:masterfrom

holdenk:SPARK-36462-allow-spark-on-kube-to-operate-without-watchers

Commits on Aug 11, 2022

Add the ability to selectively disable watching or polling for pods on Kubernetes for environments where etcd may be under a high load or otherwise not support polling/watching.
holdenk
committed
CR-feedback
holdenk
committed
Update resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala

holdenk
and
martin-g
committed
Update resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala

holdenk
and
martin-g
committed
Update resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsWatchSnapshotSourceSuite.scala

holdenk
and
martin-g
committed
Update resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsWatchSnapshotSource.scala

holdenk
and
martin-g
committed
CR feedback it's enableWatching not polling update var name to match.
holdenk
committed
Update version since we missed the boat on 3.3.0
holdenk
committed
Fix style
holdenk
committed
Move internal up above booleanConf
holdenk
committed

Commits on Aug 12, 2022

CR Feedback: add back legacy two parameter constructor, use default running sc to grab conf if no conf specified.
holdenk
committed