Bug 1771809: add PodDisruptionBudgetAtLimit alert e2e

[sallyom]

test for PodDisruptionBudgetAtLimit Alert fires when a PodDisruptionBudget with MinAvailable is created.

see also: openshift/cluster-kube-controller-manager-operator#316
https://bugzilla.redhat.com/show_bug.cgi?id=1771809#c1
/cc @soltysh

[openshift-ci-robot]

@sallyom: This pull request references Bugzilla bug 1771809, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Bug 1771809: add PodDisruptionBudget e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: sallyom
To complete the pull request process, please assign adambkaplan
You can assign the PR to them by writing /assign @adambkaplan in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• test/extended/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sallyom]

Failing bc alert is 'pending' not 'firing' -via slack discussion it's ok to check for 'pending' alert, since it will be pending for 15min. Now, have to figure out best place for this test (best place is not here where I've put it, but conveniently the extended/prometheus has set up testing alerts w/ queries already so was convenient place to start).

[sallyom]

This is so if/when this test fails cluster-kube-controller-manager-operator (API/Workloads) peeps might get the bug/issue rather than monitoring folks.

[sallyom]

Now, have to figure out best place for this test

I've created a new file test/extended/prometheus/alerts.go and have included all necessary fns to make the query in one file test/extended/prometheus/util.go, so anyone else wanting to check on a pending alert might find them.
I don't believe the 'pending' alert will interfere with the current check that no alerts are 'firing' other than Watchdog alert, since the alert will remain pending for 15 min, and as soon as the test completes (as soon as the alert is 'pending') the pod/ns is deleted.

[openshift-ci-robot]

@sallyom: This pull request references Bugzilla bug 1771809, which is valid.

Details

In response to this:

Bug 1771809: add PodDisruptionBudgetAtLimit alert e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[sallyom]

/retest

[sallyom]

closing in favor of openshift/cluster-kube-controller-manager-operator#316, openshift/library-go#602

[openshift-ci-robot]

@sallyom: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.