OCPBUGS-61063: test/extended/cli/adm_upgrade/recommend: Enable precheck and accept #30113
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
openshift-ci
bot
added
the
do-not-merge/work-in-progress
openshift-ci
bot
added
the
approved
|
Job Failure Risk Analysis for sha: fcebaa6
|
wking
force-pushed
the
test-oc-adm-upgrade-recommend-with-precheck-and-accept
branch
2 times, most recently
from
ae9e67d to
b18aac0
Compare
openshift-ci
bot
removed
the
approved
|
Job Failure Risk Analysis for sha: b18aac0
Risk analysis has seen new tests most likely introduced by this PR. New Test Risks for sha: b18aac0
New tests seen in this PR at sha: b18aac0
|
wking
force-pushed
the
test-oc-adm-upgrade-recommend-with-precheck-and-accept
branch
from
34ec4d1 to
2a74ae2
Compare
|
Job Failure Risk Analysis for sha: 2a74ae2
|
wking
force-pushed
the
test-oc-adm-upgrade-recommend-with-precheck-and-accept
branch
from
2a74ae2 to
8924956
Compare
|
Job Failure Risk Analysis for sha: 8924956
|
wking
force-pushed
the
test-oc-adm-upgrade-recommend-with-precheck-and-accept
branch
2 times, most recently
from
931699e to
d7501ee
Compare
|
Job Failure Risk Analysis for sha: d7501ee
Risk analysis has seen new tests most likely introduced by this PR. New Test Risks for sha: d7501ee
New tests seen in this PR at sha: d7501ee
|
wking
force-pushed
the
test-oc-adm-upgrade-recommend-with-precheck-and-accept
branch
2 times, most recently
from
3d796c1 to
6f8ca39
Compare
|
Job Failure Risk Analysis for sha: 6f8ca39
|
wking
force-pushed
the
test-oc-adm-upgrade-recommend-with-precheck-and-accept
branch
from
6f8ca39 to
4cbd5fe
Compare
|
Job Failure Risk Analysis for sha: 4cbd5fe
Risk analysis has seen new tests most likely introduced by this PR. New Test Risks for sha: 4cbd5fe
New tests seen in this PR at sha: 4cbd5fe
|
wking
force-pushed
the
test-oc-adm-upgrade-recommend-with-precheck-and-accept
branch
from
4cbd5fe to
b9b2c30
Compare
|
Job Failure Risk Analysis for sha: b9b2c30
Risk analysis has seen new tests most likely introduced by this PR. New Test Risks for sha: b9b2c30
New tests seen in this PR at sha: b9b2c30
|
wking
force-pushed
the
test-oc-adm-upgrade-recommend-with-precheck-and-accept
branch
from
b9b2c30 to
ccb493b
Compare
|
Job Failure Risk Analysis for sha: ccb493b
|
wking
force-pushed
the
test-oc-adm-upgrade-recommend-with-precheck-and-accept
branch
from
ccb493b to
eefe2bb
Compare
|
Job Failure Risk Analysis for sha: eefe2bb
Showing 20 of 40 jobs analysis |
wking
force-pushed
the
test-oc-adm-upgrade-recommend-with-precheck-and-accept
branch
from
eefe2bb to
93f5c58
Compare
|
Risk analysis has seen new tests most likely introduced by this PR. New Test Risks for sha: 93f5c58
New tests seen in this PR at sha: 93f5c58
|
wking
changed the title
openshift-ci-robot
removed
the
verified
Contributor
|
@wking: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
Job Failure Risk Analysis for sha: 78ac50f
|
Member
Author
|
/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-aws-ovn-single-node-serial 20 |
Contributor
|
@wking: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/6bc15d30-8b59-11f0-958c-09916b4ce3f0-0 |
petr-muller
approved these changes
Sep 8, 2025
Member
|
/lgtm |
Contributor
DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: hongkailiu, petr-muller, wking The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Member
|
The new commit 78ac50f LGTM. (Not related to this pull: I will dig into the context of https://issues.redhat.com/browse/OCPBUGS-20056 more today for my CO new rules presentation.) |
Member
|
With Petr's comment, /hold cancel I will rely on the Verified label to block the merge which is up to Trevor. |
openshift-ci
bot
removed
the
do-not-merge/hold
Member
Author
|
/verified by examining all presubmits and /payload jobs, no failures identified for these tests |
openshift-ci-robot
added
the
verified
|
@wking: This PR has been marked as verified by DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-merge-bot
bot
merged commit 755c0d2
into
openshift:main
44 of 47 checks passed
|
@wking: Jira Issue OCPBUGS-61063: Some pull requests linked via external trackers have merged: The following pull request, linked via external tracker, has not merged:
All associated pull requests must be merged or unlinked from the Jira bug in order for it to move to the next state. Once unlinked, request a bug refresh with Jira Issue OCPBUGS-61063 has not been moved to the MODIFIED state. This PR is marked as verified. If the remaining PRs listed above are marked as verified before merging, the issue will automatically be moved to VERIFIED after all of the changes from the PRs are available in an accepted nightly payload. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
wking
deleted the
test-oc-adm-upgrade-recommend-with-precheck-and-accept
branch
Member
Author
|
/cherrypick release-4.20 |
openshift-cherrypick-robot
pushed a commit
to openshift-cherrypick-robot/origin
that referenced
this pull request
Sep 15, 2025
…False In 44cd78a (test/extended/cli/adm_upgrade/recommend: Account for Upgradeable=False, 2025-08-05, openshift#30113), I'd updated the regular expression to accept: Reason: MultipleReasons for clusters that had both Upgradeable=False and conditional risks going on. In 7724a75 (test/extended/cli/adm_upgrade/recommend: Trust the ingress CA, 2025-08-14, openshift#30113), I extended the regular expression to cover: Reason: accepted TestRiskA via ConditionalUpdateRisk But I hadn't thought through the MultipleReasons case, and this commit catches us up to tech-preview serial output like [1]: ... Reason: accepted MultipleReasons via ConditionalUpdateRisk Message: Cluster operator config-operator should not be upgraded between minor versions: FeatureGatesUpgradeable: "TechPreviewNoUpgrade" does not allow updates This is a test risk. https://example.com/testRiskA ... [1]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/30113/pull-ci-openshift-origin-main-e2e-gcp-ovn-techpreview-serial-1of2/1963672599955247104
openshift-cherrypick-robot
pushed a commit
to openshift-cherrypick-robot/origin
that referenced
this pull request
Sep 15, 2025
A depressing amount of the time in single-node testing, the
authentication ClusterOperator is Available=False, and causes the
'recommend' command to error if the 'Failing' risk is not accepted.
For example, in this aggregation run's 20 attempts [1]:
* 9 passed.
* three failed the '--version' test-case on 'authentication' 'Available=False' [2,3,4].
* two failed on build-cluster registry 500s [5,6].
* two failed on 'prometheus-operator' watch requests [7,8].
* one failed on a 'context deadline exceeded' out of
'runUpdateService' [9], with 'Failed to pull image...authentication
required' issues trying to get the 'tools' image.
* one failed on authentication Pod restarts [10].
* one failed on an un-excepted 'authentication' 'Available=False'
('OAuthServerDeployment_NoPod') [11].
* one failed on an unexpected successful return in an
TestImageStreamTagsAdmission test-case [12].
So the auth functionality is pretty flaky in these single-node
clusters under serial-suite load. Ideally [13] can get addressed or
the auth component can otherwise get firmed up, but until then, this
commit softens our logic to allow that kind of ClusterOperator
Available=False (which gets bubbled up as ClusterVersion
Failing=True), without failing our new test-case.
[1]: openshift#30113 (comment)
[2]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055948225941504
[3]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055951426195456
[4]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055945973600256
[5]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055945063436288#1:build-log.txt%3A32
[6]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055945512226816#1:build-log.txt%3A43
[7]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055950948044800
[8]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055948683120640
[9]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055952327970816
[10]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055953233940480
[11]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055947772956672
[12]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055952797732864
[13]: https://issues.redhat.com/browse/OCPBUGS-20056
|
@wking: new pull request created: #30260 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Contributor
|
Fix included in accepted release 4.21.0-0.nightly-2025-09-11-143911 |
ShudiLi
pushed a commit
to ShudiLi/origin
that referenced
this pull request
Dec 11, 2025
…False In 44cd78a (test/extended/cli/adm_upgrade/recommend: Account for Upgradeable=False, 2025-08-05, openshift#30113), I'd updated the regular expression to accept: Reason: MultipleReasons for clusters that had both Upgradeable=False and conditional risks going on. In 7724a75 (test/extended/cli/adm_upgrade/recommend: Trust the ingress CA, 2025-08-14, openshift#30113), I extended the regular expression to cover: Reason: accepted TestRiskA via ConditionalUpdateRisk But I hadn't thought through the MultipleReasons case, and this commit catches us up to tech-preview serial output like [1]: ... Reason: accepted MultipleReasons via ConditionalUpdateRisk Message: Cluster operator config-operator should not be upgraded between minor versions: FeatureGatesUpgradeable: "TechPreviewNoUpgrade" does not allow updates This is a test risk. https://example.com/testRiskA ... [1]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/30113/pull-ci-openshift-origin-main-e2e-gcp-ovn-techpreview-serial-1of2/1963672599955247104
ShudiLi
pushed a commit
to ShudiLi/origin
that referenced
this pull request
Dec 11, 2025
A depressing amount of the time in single-node testing, the
authentication ClusterOperator is Available=False, and causes the
'recommend' command to error if the 'Failing' risk is not accepted.
For example, in this aggregation run's 20 attempts [1]:
* 9 passed.
* three failed the '--version' test-case on 'authentication' 'Available=False' [2,3,4].
* two failed on build-cluster registry 500s [5,6].
* two failed on 'prometheus-operator' watch requests [7,8].
* one failed on a 'context deadline exceeded' out of
'runUpdateService' [9], with 'Failed to pull image...authentication
required' issues trying to get the 'tools' image.
* one failed on authentication Pod restarts [10].
* one failed on an un-excepted 'authentication' 'Available=False'
('OAuthServerDeployment_NoPod') [11].
* one failed on an unexpected successful return in an
TestImageStreamTagsAdmission test-case [12].
So the auth functionality is pretty flaky in these single-node
clusters under serial-suite load. Ideally [13] can get addressed or
the auth component can otherwise get firmed up, but until then, this
commit softens our logic to allow that kind of ClusterOperator
Available=False (which gets bubbled up as ClusterVersion
Failing=True), without failing our new test-case.
[1]: openshift#30113 (comment)
[2]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055948225941504
[3]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055951426195456
[4]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055945973600256
[5]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055945063436288#1:build-log.txt%3A32
[6]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055945512226816#1:build-log.txt%3A43
[7]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055950948044800
[8]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055948683120640
[9]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055952327970816
[10]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055953233940480
[11]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055947772956672
[12]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/openshift-origin-30113-nightly-4.21-e2e-aws-ovn-single-node-serial/1964055952797732864
[13]: https://issues.redhat.com/browse/OCPBUGS-20056
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These will probably not pass without more work, but checking to see how far away we are.