Skip to content

[SPARK-26015][K8S] Set a default UID for Spark on K8S Images #23017

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rvesse wants to merge 8 commits into from
Closed

[SPARK-26015][K8S] Set a default UID for Spark on K8S Images #23017

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
26697fc
[SPARK-26015][K8S] Set a default UID for Spark on K8S Images
rvesse Nov 12, 2018
6680c13
[SPARK-26015][K8S] Fix broken client mode test
rvesse Nov 15, 2018
11419e3
[SPARK-26015][K8S] If SPARK_USER_NAME is present use it
rvesse Nov 15, 2018
db1e83a
[SPARK-26015][K8S] Address PR Comments
rvesse Nov 16, 2018
fd37bdd
[SPARK-26015][K8S] Fix line breaks
rvesse Nov 16, 2018
6c6232e
[SPARK-26015][K8S] Updates docs for USER changes
rvesse Nov 20, 2018
72fb30c
[SPARK-26015][K8S] Fix incorrectly removed import
rvesse Nov 20, 2018
8ab866b
[SPARK-26015][K8S] Fix up changes to align with master changes
rvesse Nov 29, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[SPARK-26015][K8S] Fix broken client mode test 
The client mode test was incorrectly overriding the entry point of the
image so didn't benefit from the logic that set up the /etc/passwd entry
for the container UID resulting in no home directory and a failed Ivy
setup as a result
  • Loading branch information
@rvesse
rvesse committed Nov 29, 2018
commit 6680c134043c922ab2e613177897dae3a0b6d672
7 changes: 4 additions & 3 deletions ...sts/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/ClientModeTestsSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@
package org.apache.spark.deploy.k8s.integrationtest

import org.scalatest.concurrent.Eventually
import scala.collection.JavaConverters._

import org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.{k8sTestTag, INTERVAL, TIMEOUT}
import org.scalatest.Tag

private[spark] trait ClientModeTestsSuite { k8sSuite: KubernetesSuite =>

test("Run in client mode.", k8sTestTag) {
test("Run in client mode.", k8sTestTag, Tag("k8s-client-mode")) {
val labels = Map("spark-app-selector" -> driverPodName)
val driverPort = 7077
val blockManagerPort = 10000
Expand Down Expand Up @@ -62,11 +62,12 @@ private[spark] trait ClientModeTestsSuite { k8sSuite: KubernetesSuite =>
.endMetadata()
.withNewSpec()
.withServiceAccountName(kubernetesTestComponents.serviceAccountName)
.withRestartPolicy("Never")
.addNewContainer()
.withName("spark-example")
.withImage(image)
.withImagePullPolicy("IfNotPresent")
.withCommand("/opt/spark/bin/run-example")
.addToArgs("/opt/spark/bin/run-example")
.addToArgs("--master", s"k8s://https://kubernetes.default.svc")
.addToArgs("--deploy-mode", "client")
.addToArgs("--conf", s"spark.kubernetes.container.image=$image")
Expand Down