[SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2 #24002

Jeffwan · 2019-03-07T06:39:26Z

What changes were proposed in this pull request?

#23814 was reverted because of Jenkins integration tests failure. After minikube upgrade, Kubernetes client SDK v1.4.2 work with kubernetes v1.13. We can bring this change back.

Reference:
Bump Kubernetes Client Version to 4.1.2
Original PR against master
Kubernetes client upgrade for Spark 2.4

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

Unit Tests:

All tests passed.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.0.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [  2.343 s]
[INFO] Spark Project Tags ................................. SUCCESS [  2.039 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 12.714 s]
[INFO] Spark Project Local DB ............................. SUCCESS [  2.185 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 38.154 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [  7.989 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [  2.297 s]
[INFO] Spark Project Launcher ............................. SUCCESS [  2.813 s]
[INFO] Spark Project Core ................................. SUCCESS [38:03 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [  3.848 s]
[INFO] Spark Project GraphX ............................... SUCCESS [ 56.084 s]
[INFO] Spark Project Streaming ............................ SUCCESS [04:58 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [06:39 min]
[INFO] Spark Project SQL .................................. SUCCESS [37:12 min]
[INFO] Spark Project ML Library ........................... SUCCESS [18:59 min]
[INFO] Spark Project Tools ................................ SUCCESS [  0.767 s]
[INFO] Spark Project Hive ................................. SUCCESS [33:45 min]
[INFO] Spark Project REPL ................................. SUCCESS [01:14 min]
[INFO] Spark Project Assembly ............................. SUCCESS [  1.444 s]
[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [01:12 min]
[INFO] Kafka 0.10+ Token Provider for Streaming ........... SUCCESS [  6.719 s]
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS [07:00 min]
[INFO] Spark Project Examples ............................. SUCCESS [ 21.805 s]
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [  0.906 s]
[INFO] Spark Avro ......................................... SUCCESS [ 50.486 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:32 h
[INFO] Finished at: 2019-03-07T08:39:34Z
[INFO] ------------------------------------------------------------------------

Please review http://spark.apache.org/contributing.html before opening a pull request.

Jeffwan · 2019-03-07T06:41:27Z

Please hold this PR until @shaneknapp confirms. Not sure if minikube get updated on Jenkins side.

shaneknapp · 2019-03-07T18:09:21Z

yep, gonna run these changes manually against the k8s integration tests now. main conversation taking place here: https://issues.apache.org/jira/browse/SPARK-26742

shaneknapp · 2019-03-07T18:47:12Z

testing against the 4.1.2 client passed. after i complete the upgrade on the ubuntu workers, i'll approve and trigger this build.

see also: https://issues.apache.org/jira/browse/SPARK-26742?focusedCommentId=16787090&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16787090

Jeffwan · 2019-03-07T18:52:26Z

I also successfully finish integration test on v1.10.13 ! I will also try v1.11.7 v1.12.6 and v1.13.3

Run completed in 7 minutes, 31 seconds.
Total number of tests run: 15
Suites: completed 2, aborted 0
Tests: succeeded 15, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.0.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [  4.647 s]
[INFO] Spark Project Tags ................................. SUCCESS [  3.894 s]
[INFO] Spark Project Local DB ............................. SUCCESS [  2.867 s]
[INFO] Spark Project Networking ........................... SUCCESS [  5.506 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [  3.178 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [  3.026 s]
[INFO] Spark Project Launcher ............................. SUCCESS [  3.903 s]
[INFO] Spark Project Core ................................. SUCCESS [ 32.987 s]
[INFO] Spark Project Kubernetes Integration Tests ......... SUCCESS [08:28 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  09:28 min
[INFO] Finished at: 2019-03-07T10:51:01-08:00
[INFO] ------------------------------------------------------------------------

shaneknapp · 2019-03-07T19:33:39Z

ok to test

SparkQA · 2019-03-07T19:45:32Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/8606/

SparkQA · 2019-03-07T19:58:35Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/8606/

Jeffwan · 2019-03-07T20:07:52Z

java.net.ProtocolException: Expected HTTP 101 response but was '404 Not Found'
	at okhttp3.internal.ws.RealWebSocket.checkResponse(RealWebSocket.java:229)
	at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:196)
	at okhttp3.RealCall$AsyncCall.execute(RealCall.java:206)
	at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
19/03/07 11:53:57.968 OkHttp https://192.168.39.69:8443/... ERROR ExecWebSocketListener: Exec Failure: HTTP:404. Message:404 page not found

Weird, Looks like that's the same error before upgrade? I didn't see this problem in local testing.

Jeffwan · 2019-03-07T20:21:57Z

@shaneknapp
From raw logs, seems kubernetes cluster is v1.10.0?

+ /home/jenkins/bin/session_lock_resource.py minikube
+ /home/jenkins/bin/post_github_pr_comment.py
Attempting to post to Github...
 > Post successful.
+ minikube --vm-driver=kvm2 start --memory 6000 --cpus 8
Starting local Kubernetes v1.10.0 cluster...

SparkQA · 2019-03-08T00:38:25Z

Test build #103152 has finished for PR 24002 at commit d4a155c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

shaneknapp · 2019-03-08T18:55:01Z

yeah, the k8s/minikube test is supposed to fail as i haven't upgraded those workers yet... i just wanted to make sure that the regular PRB build passed.

Jeffwan · 2019-03-08T18:59:06Z

@shaneknapp Got you. I don't know they're two different flows. Good to know. :D

Jeffwan · 2019-03-11T23:19:35Z

@shaneknapp @gatorsmile Look like changes against 2.4 has been closed. Is this one ready to merge?

vanzin · 2019-03-11T23:40:28Z

This can't be merged until the k8s integration tests pass. That would mean they would fail on every other k8s-related PR, and we can't have that.

Jeffwan · 2019-03-11T23:56:05Z

@vanzin Thanks. I am a little bit confused about test status. It doesn't show any status on my end.

PR triggered job fails.
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/8606/

But seems it get success later by manual kicked test by @shaneknapp ?
Test build #103152 has finished for PR 24002 at commit d4a155c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

@shaneknapp could you take a look?

shaneknapp · 2019-03-12T00:03:00Z

@vanzin -- these tests won't pass until i upgrade the minikube + k8s versions on the ubuntu workers (which is actually ready to go!). i've tested this PR manually and everything works.

regarding #23993 -- i see it's closed but not merged? i checked out the latest spark-2.4.1-rc8 tag and none of the files changed in the 2.4.1 PR show the 4.1.2 k8s client.

so...... i guess we're waiting for the spark 2.4.1 cut?

vanzin · 2019-03-12T01:45:17Z

I'm just making a point that this PR should not be merged until the actual, automatic, k8s it run passes.

Unless you're willing to manually monitor all k8s-related PRs and manually run integration tests on them, and report the results.

shaneknapp · 2019-03-12T01:59:53Z

I'm just making a point that this PR should not be merged until the actual, automatic, k8s it run passes.

Unless you're willing to manually monitor all k8s-related PRs and manually run integration tests on them, and report the results.

ah, gotcha. :)

it's a matter of timing this w/the 2.4.1 release. do we do the infra/k8s/minikube upgrade before or after the code freezes or wait until afterwards.

there was a lot of traffic on dev@ about the 2.4.1 RCs, but this bit may have been lost in the noise. i'll send out an email later and see what people's thoughts are.

Jeffwan · 2019-03-12T05:05:00Z

@shaneknapp @vanzin Thanks for explanation. Now I get it.

vanzin · 2019-03-13T22:04:11Z

Merging to master (while Shane messes with jenkins).

[SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2

d4a155c

vanzin closed this in 2d0b7cf Mar 13, 2019

Jeffwan deleted the update_k8s_sdk_master branch March 15, 2019 19:18

[SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2 #24002

[SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2 #24002

Uh oh!

Conversation

Jeffwan commented Mar 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

Jeffwan commented Mar 7, 2019

Uh oh!

shaneknapp commented Mar 7, 2019

Uh oh!

shaneknapp commented Mar 7, 2019

Uh oh!

Jeffwan commented Mar 7, 2019

Uh oh!

shaneknapp commented Mar 7, 2019

Uh oh!

SparkQA commented Mar 7, 2019

Uh oh!

SparkQA commented Mar 7, 2019

Uh oh!

Jeffwan commented Mar 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jeffwan commented Mar 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SparkQA commented Mar 8, 2019

Uh oh!

shaneknapp commented Mar 8, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jeffwan commented Mar 8, 2019

Uh oh!

Jeffwan commented Mar 11, 2019

Uh oh!

vanzin commented Mar 11, 2019

Uh oh!

Jeffwan commented Mar 11, 2019

Uh oh!

shaneknapp commented Mar 12, 2019

Uh oh!

vanzin commented Mar 12, 2019

Uh oh!

shaneknapp commented Mar 12, 2019

Uh oh!

Jeffwan commented Mar 12, 2019

Uh oh!

vanzin commented Mar 13, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Jeffwan commented Mar 7, 2019 •

edited

Loading

Jeffwan commented Mar 7, 2019 •

edited

Loading

Jeffwan commented Mar 7, 2019 •

edited

Loading

shaneknapp commented Mar 8, 2019 •

edited

Loading