Skip to content

OCPBUGS-61976: fix(azure-metrics): add retry logic with exponential backoff for loadbalancer lookup #30278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
jparrill wants to merge 1 commit into
base: main
Choose a base branch
from
Open

OCPBUGS-61976: fix(azure-metrics): add retry logic with exponential backoff for loadbalancer lookup #30278

jparrill wants to merge 1 commit into from

Conversation

@jparrill
Copy link
Contributor

@jparrill jparrill commented Sep 19, 2025 •
edited
Loading

What this PR does / why we need it

Azure AKS conformance tests were failing because the azure-metrics-collector
was attempting to access load balancers before they were fully created,
resulting in 404 ResourceNotFound errors.

This change adds retry logic with exponential backoff to the getLoadBalancerID
function:

  • Retry up to 10 times for 404 errors only
  • Initial backoff of 10s, doubling each retry up to 5min max
  • Total max wait time ~20 minutes
  • Proper context cancellation handling
  • Detailed logging for debugging

The fix ensures the metrics collector waits for load balancer creation
instead of failing immediately on timing issues.

Which issue(s) this PR fixes

Additional Info

Job try 1 try 2 try 3 try 4
periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-azure-aks-ovn-conformance 🟢 ? ? ?
periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-azure-aks-ovn-conformance 🟢 ? ? ?
periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-aks 🟢 ? ? ?
periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-aks 🟢 ? ? ?

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 19, 2025
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 19, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@jparrill
Copy link
Contributor Author

jparrill commented Sep 19, 2025

/test unit

@jparrill
Copy link
Contributor Author

jparrill commented Sep 19, 2025

/test verify

@jparrill
Copy link
Contributor Author

jparrill commented Sep 19, 2025

/test e2e-azure

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 19, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-azure-aks-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/a65e5280-955a-11f0-9c55-15d55240b00c-0

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 19, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-azure-aks-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/a9663cae-955a-11f0-85a6-3df681f70f55-0

@jparrill
Copy link
Contributor Author

jparrill commented Sep 19, 2025

/payload-job periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-azure-aks-ovn-conformance

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 19, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-azure-aks-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/ae7da330-955a-11f0-8893-f3540b276085-0

@jparrill
Copy link
Contributor Author

jparrill commented Sep 22, 2025

/test e2e-azure

@jparrill
Copy link
Contributor Author

jparrill commented Sep 22, 2025

/payload-job periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-azure-aks-ovn-conformance

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 22, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-azure-aks-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/649155e0-978b-11f0-8b53-b976e8dd9b68-0

@jparrill
Copy link
Contributor Author

jparrill commented Sep 22, 2025

/payload-job periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-azure-aks-ovn-conformance

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 22, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-azure-aks-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/0a2b4970-978c-11f0-9d3c-513df797b711-0

@jparrill jparrill marked this pull request as ready for review September 22, 2025 08:42
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 22, 2025
@jparrill
Copy link
Contributor Author

jparrill commented Sep 22, 2025

/payload periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-aks

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 22, 2025

@jparrill: it appears that you have attempted to use some version of the payload command, but your comment was incorrectly formatted and cannot be acted upon. See the docs for usage info.

@jparrill
Copy link
Contributor Author

jparrill commented Sep 22, 2025

/payload-job periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-aks

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 22, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-aks

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/cb48dc40-9790-11f0-8d01-b886ad32595f-0

@jparrill
Copy link
Contributor Author

jparrill commented Sep 22, 2025

/unassign @deads2k

@jparrill
Copy link
Contributor Author

jparrill commented Sep 22, 2025

/assign @bryan-cox

@jparrill
Copy link
Contributor Author

jparrill commented Sep 22, 2025

/assign @sjenning

@jparrill
Copy link
Contributor Author

jparrill commented Sep 22, 2025

/unassign @p0lyn0mial

@jparrill
Copy link
Contributor Author

jparrill commented Sep 22, 2025

/uncc @p0lyn0mial

@openshift-ci openshift-ci bot removed the request for review from p0lyn0mial September 22, 2025 10:14
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 23, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-azure-aks-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/79d17600-9863-11f0-9f22-6a1bc45aa370-0

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 23, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-azure-aks-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/7f18ee90-9863-11f0-9dd3-db03123ef720-0

@jparrill
Copy link
Contributor Author

jparrill commented Sep 23, 2025

/payload-job periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-aks

@jparrill
Copy link
Contributor Author

jparrill commented Sep 23, 2025

/payload-job periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-aks

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 23, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-aks

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/8a399860-9863-11f0-8e5f-3ff9deaf6ded-0

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 23, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-aks

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/8dbbe150-9863-11f0-9145-34b833cb87f9-0

@jparrill
Copy link
Contributor Author

jparrill commented Sep 23, 2025

There are periodics failing because of this: openshift/hypershift#6872

@sjenning
Copy link
Contributor

sjenning commented Sep 23, 2025

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 23, 2025
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 23, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jparrill, sjenning
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-trt
Copy link

openshift-trt bot commented Sep 23, 2025

Job Failure Risk Analysis for sha: c76c042

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-ovn Medium
Job run should complete before timeout
This test has passed 88.11% of 4812 runs on release 4.21 [Overall] in the last week.
pull-ci-openshift-origin-main-e2e-aws-ovn-kube-apiserver-rollout IncompleteTests
Tests for this run (21) are below the historical average (1498): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (21) are below the historical average (2599): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@openshift-trt
Copy link

openshift-trt bot commented Sep 23, 2025

Job Failure Risk Analysis for sha: c76c042

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-disruptive Low
Job run should complete before timeout
This test has passed 38.46% of 13 runs on release 4.21 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:hidden Network:ovn NetworkStack:ipv4 Owner:eng Platform:aws Procedure:none SecurityMode:default Topology:ha Upgrade:micro-downgrade] in the last week.
pull-ci-openshift-origin-main-e2e-aws-ovn Medium
Job run should complete before timeout
This test has passed 88.11% of 4812 runs on release 4.21 [Overall] in the last week.
pull-ci-openshift-origin-main-e2e-aws-ovn-kube-apiserver-rollout IncompleteTests
Tests for this run (21) are below the historical average (1498): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (21) are below the historical average (2599): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@jparrill
Copy link
Contributor Author

jparrill commented Sep 24, 2025

/payload-job periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-aks

@jparrill
Copy link
Contributor Author

jparrill commented Sep 24, 2025

/payload-job periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-azure-aks-ovn-conformance

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 24, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-aks

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/a8d7b8a0-9918-11f0-9a5c-dc8c920a72ee-0

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 24, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-azure-aks-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/ad0c88b0-9918-11f0-81a9-15908416e332-0

@jparrill
Copy link
Contributor Author

jparrill commented Sep 24, 2025

/payload-job periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-azure-aks-ovn-conformance

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 24, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.21-periodics-e2e-azure-aks-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/b1e95890-9918-11f0-97b9-0f33933e851b-0

@jparrill
Copy link
Contributor Author

jparrill commented Sep 24, 2025

/payload-job periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-azure-aks-ovn-conformance

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 24, 2025

@jparrill: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-4.20-periodics-e2e-azure-aks-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/b539e320-9918-11f0-97c9-0cbc1f5822e1-0

@jparrill
Copy link
Contributor Author

jparrill commented Sep 24, 2025

/retest

@openshift-trt
Copy link

openshift-trt bot commented Sep 24, 2025

Job Failure Risk Analysis for sha: c76c042

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-cgroupsv2 IncompleteTests
Tests for this run (25) are below the historical average (2568): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-edge-zones IncompleteTests
Tests for this run (104) are below the historical average (2605): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-kube-apiserver-rollout IncompleteTests
Tests for this run (103) are below the historical average (1453): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (103) are below the historical average (2429): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-upgrade IncompleteTests
Tests for this run (105) are below the historical average (1555): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-proxy Medium
[sig-network] Services should be rejected for evicted pods (no endpoints exist) [Suite:openshift/conformance/parallel] [Suite:k8s]
This test has passed 92.13% of 2160 runs on release 4.21 [Overall] in the last week.
pull-ci-openshift-origin-main-okd-scos-e2e-aws-ovn IncompleteTests
Tests for this run (20) are below the historical average (2440): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@openshift-trt
Copy link

openshift-trt bot commented Sep 24, 2025

Job Failure Risk Analysis for sha: c76c042

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-cgroupsv2 IncompleteTests
Tests for this run (25) are below the historical average (2568): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-edge-zones IncompleteTests
Tests for this run (104) are below the historical average (2605): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-kube-apiserver-rollout IncompleteTests
Tests for this run (103) are below the historical average (1453): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (103) are below the historical average (2413): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-upgrade IncompleteTests
Tests for this run (105) are below the historical average (1555): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-proxy Medium
Job run should complete before timeout
This test has passed 83.33% of 6 runs on release 4.21 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:standard Network:ovn NetworkStack:ipv4 Owner:eng Platform:aws Procedure:none SecurityMode:default Topology:ha Upgrade:none] in the last week.
pull-ci-openshift-origin-main-okd-scos-e2e-aws-ovn IncompleteTests
Tests for this run (20) are below the historical average (2422): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@openshift-trt
Copy link

openshift-trt bot commented Sep 24, 2025

Job Failure Risk Analysis for sha: c76c042

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-cgroupsv2 IncompleteTests
Tests for this run (25) are below the historical average (2549): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-edge-zones IncompleteTests
Tests for this run (104) are below the historical average (2605): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-kube-apiserver-rollout IncompleteTests
Tests for this run (103) are below the historical average (1453): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (103) are below the historical average (2413): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-upgrade IncompleteTests
Tests for this run (105) are below the historical average (1555): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-proxy Medium
Job run should complete before timeout
This test has passed 83.33% of 6 runs on release 4.21 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:standard Network:ovn NetworkStack:ipv4 Owner:eng Platform:aws Procedure:none SecurityMode:default Topology:ha Upgrade:none] in the last week.
pull-ci-openshift-origin-main-okd-scos-e2e-aws-ovn IncompleteTests
Tests for this run (20) are below the historical average (2422): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@openshift-trt
Copy link

openshift-trt bot commented Oct 3, 2025

Job Failure Risk Analysis for sha: c76c042

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-cgroupsv2 IncompleteTests
Tests for this run (25) are below the historical average (1834): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-edge-zones IncompleteTests
Tests for this run (104) are below the historical average (1967): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-kube-apiserver-rollout IncompleteTests
Tests for this run (103) are below the historical average (1034): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (103) are below the historical average (1789): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-upgrade IncompleteTests
Tests for this run (105) are below the historical average (1182): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-okd-scos-e2e-aws-ovn IncompleteTests
Tests for this run (20) are below the historical average (1542): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@openshift-trt
Copy link

openshift-trt bot commented Oct 3, 2025

Job Failure Risk Analysis for sha: c76c042

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-cgroupsv2 IncompleteTests
Tests for this run (25) are below the historical average (1834): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-edge-zones IncompleteTests
Tests for this run (104) are below the historical average (1967): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-kube-apiserver-rollout IncompleteTests
Tests for this run (103) are below the historical average (1034): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node IncompleteTests
Tests for this run (103) are below the historical average (1805): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-upgrade IncompleteTests
Tests for this run (105) are below the historical average (1209): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-okd-scos-e2e-aws-ovn IncompleteTests
Tests for this run (20) are below the historical average (1532): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@openshift-trt
Copy link

openshift-trt bot commented Nov 4, 2025

Job Failure Risk Analysis for sha: c76c042

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-proxy Medium
Job run should complete before timeout
This test has passed 90.42% of 4322 runs on release 4.21 [Overall] in the last week.
pull-ci-openshift-origin-main-okd-scos-e2e-aws-ovn IncompleteTests
Tests for this run (20) are below the historical average (3057): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@openshift-trt
Copy link

openshift-trt bot commented Nov 4, 2025

Job Failure Risk Analysis for sha: c76c042

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-proxy Medium
Job run should complete before timeout
This test has passed 90.42% of 4334 runs on release 4.21 [Overall] in the last week.
pull-ci-openshift-origin-main-okd-scos-e2e-aws-ovn IncompleteTests
Tests for this run (20) are below the historical average (3057): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Nov 18, 2025

@jparrill: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-metal-ipi-virtualmedia c76c042 link false /test e2e-metal-ipi-virtualmedia
ci/prow/okd-scos-e2e-aws-ovn c76c042 link false /test okd-scos-e2e-aws-ovn
ci/prow/e2e-aws-ovn-cgroupsv2 c76c042 link false /test e2e-aws-ovn-cgroupsv2
ci/prow/e2e-aws-ovn-single-node-upgrade c76c042 link false /test e2e-aws-ovn-single-node-upgrade
ci/prow/e2e-aws-ovn-single-node c76c042 link false /test e2e-aws-ovn-single-node
ci/prow/e2e-metal-ipi-ovn-kube-apiserver-rollout c76c042 link false /test e2e-metal-ipi-ovn-kube-apiserver-rollout
ci/prow/e2e-aws-ovn-edge-zones c76c042 link false /test e2e-aws-ovn-edge-zones
ci/prow/e2e-openstack-ovn c76c042 link false /test e2e-openstack-ovn
ci/prow/e2e-aws-ovn-kube-apiserver-rollout c76c042 link false /test e2e-aws-ovn-kube-apiserver-rollout
ci/prow/e2e-aws-proxy c76c042 link false /test e2e-aws-proxy
ci/prow/e2e-aws-ovn-single-node-serial c76c042 link false /test e2e-aws-ovn-single-node-serial
ci/prow/e2e-aws-ovn-upgrade c76c042 link false /test e2e-aws-ovn-upgrade
ci/prow/e2e-aws-ovn c76c042 link false /test e2e-aws-ovn
ci/prow/e2e-metal-ipi-serial-ovn-ipv6-1of2 c76c042 link false /test e2e-metal-ipi-serial-ovn-ipv6-1of2
ci/prow/e2e-aws-ovn-serial-2of2 c76c042 link true /test e2e-aws-ovn-serial-2of2
ci/prow/e2e-aws-disruptive c76c042 link false /test e2e-aws-disruptive
ci/prow/e2e-aws-ovn-fips c76c042 link true /test e2e-aws-ovn-fips
ci/prow/e2e-gcp-ovn c76c042 link true /test e2e-gcp-ovn
ci/prow/e2e-aws-ovn-serial-1of2 c76c042 link true /test e2e-aws-ovn-serial-1of2
ci/prow/e2e-aws-csi c76c042 link true /test e2e-aws-csi
ci/prow/e2e-gcp-csi c76c042 link true /test e2e-gcp-csi
ci/prow/go-verify-deps c76c042 link true /test go-verify-deps
ci/prow/e2e-aws-ovn-microshift-serial c76c042 link true /test e2e-aws-ovn-microshift-serial
ci/prow/e2e-metal-ipi-ovn-ipv6 c76c042 link true /test e2e-metal-ipi-ovn-ipv6

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-trt
Copy link

openshift-trt bot commented Nov 18, 2025

Job Failure Risk Analysis for sha: c76c042

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-cgroupsv2 Medium
[sig-sippy] infrastructure should work
This test has passed 85.37% of 5808 runs on release 4.21 [Overall] in the last week.
pull-ci-openshift-origin-main-e2e-aws-ovn-edge-zones IncompleteTests
Tests for this run (104) are below the historical average (3898): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node-upgrade Medium
Job run should complete before timeout
This test has passed 91.48% of 5293 runs on release 4.21 [Overall] in the last week.
pull-ci-openshift-origin-main-e2e-aws-proxy Medium
Job run should complete before timeout
This test has passed 91.48% of 5293 runs on release 4.21 [Overall] in the last week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@sjenning sjenning

@bryan-cox bryan-cox

Labels

jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jparrill @openshift-ci-robot @sjenning @bryan-cox