Bug 1946479: prevent pod deployment deadlock due to custom SA projected volume injection #585
Conversation
…ection In an upgrade scenario: 1. a kube apiserver updates, any pod request that passes its admission is injected with SA projected volume 2. OpenShift adds a configMap "openshift-service-ca.crt" among the volume sources to the projected volume 3. Any pod that gets created now will be stuck until an updated version of KCM gets started 4. If CVO pod gets deleted at this point, the whole upgrade might get stuck Prevent the deadlock by manually configuring the projected volume. This can be removed in 4.9 when even the older KCM contains the code creating the openshift-service-ca.crt CMs.
openshift-ci
bot
added
bugzilla/severity-high
|
@stlaz: This pull request references Bugzilla bug 1946479, which is valid. The bug has been updated to refer to the pull request using the external bug tracker. 3 validation(s) were run on this bug
Requesting review from QA contact: DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: stlaz, wking The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
added
the
approved
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
@stlaz Does this projection rely on features of either a 4.7 or 4.8 kubelet? Just asking in case this is something that would inhibit a 4.6 kubelet from working properly with a 4.8 cluster. |
|
This relies solely on features from a 4.8 kube-controller-manager and kube-apiserver, kubelet is not involved. edit: no new features in kubelet are involved |
|
@stlaz: Some pull requests linked via external trackers have merged: The following pull requests linked via external trackers have not merged: These pull request must merge or be unlinked from the Bugzilla bug in order for it to move to the next state. Once unlinked, request a bug refresh with Bugzilla bug 1946479 has not been moved to the MODIFIED state. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
…lume injection" This reverts commit 9bde7bb, openshift#585. It was a temporary workaround in 4.8 to bridge a gap between Kube-API-server and Kube-controller-manager during 4.7 -> 4.8 updates. Once the cluster has reconciled 4.8, we will no longer need the crutch. And 4.8 has now forked off master, so we can revert now, and that revert will go out in 4.9.
6 participants
In an upgrade scenario:
is injected with SA projected volume
volume sources to the projected volume
version of KCM gets started
get stuck
Prevent the deadlock by manually configuring the projected volume.
This can be removed in 4.9 when even the older KCM contains the code
creating the openshift-service-ca.crt CMs.