Skip to content

ci-operator/config/openshift/cluster-monitoring-operator: Generic e2e-aws -> e2e for 4.6+ #10377

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
openshift-merge-robot merged 1 commit into from
Aug 12, 2020
Merged

ci-operator/config/openshift/cluster-monitoring-operator: Generic e2e-aws -> e2e for 4.6+ #10377

openshift-merge-robot merged 1 commit into from
Aug 12, 2020

Conversation

@wking
Copy link
Member

@wking wking commented Jul 21, 2020

Following the pattern from e7bb102 (#10152), the "e2e-aws" -> "e2e" change declares the monitoring-operator presubmits to be platform-agnostic.

The AWS -> GCP change takes advantage of the platform-agnosticism to shift CI load from AWS (where we're currently pegging Boskos lease capacity) to GCP (where we have some spare lease capacity).

Generated by manually changing ci-operator/config/... and then running:

$ make update

Process docs in flight with #10166.

@wking wking force-pushed the platform-agnostic-monitoring-presubmits branch from ed2dcd9 to 7fc9554 Compare July 21, 2020 20:38
@lilic
Copy link
Contributor

lilic commented Jul 22, 2020

cc @openshift/openshift-team-monitoring

/retest

@simonpasquier
Copy link
Contributor

simonpasquier commented Jul 22, 2020

Looking at https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_release/10377/rehearse-10377-pull-ci-openshift-cluster-monitoring-operator-master-e2e-operator/1285834842889523200, it fails with:

 2020/07/22 07:42:17 cannot find token for openshift-monitoring/cluster-monitoring-operator-e2e service account

Not sure what's different in GCP from AWS that triggers this issue...

@simonpasquier simonpasquier mentioned this pull request Jul 22, 2020
Merged
2 tasks
@simonpasquier
Copy link
Contributor

simonpasquier commented Jul 22, 2020

After investigating the cluster-monitoring-operator e2e test code, we might have a race if Kubernetes doesn't populate the service account's secret quickly enough. I've submitted openshift/cluster-monitoring-operator#876 so the test framework retries if it can't find the secret immediately.

@wking
Copy link
Member Author

wking commented Jul 23, 2020

openshift/cluster-monitoring-operator#876 landed.

/retest

@simonpasquier
Copy link
Contributor

simonpasquier commented Jul 24, 2020

/retest

@simonpasquier
Copy link
Contributor

simonpasquier commented Jul 24, 2020

I suspect that we have an issue because we rely on the gp2 storage class in our e2e tests:

https://github.com/openshift/cluster-monitoring-operator/blob/e1caabda745caba6e4784095aebd75f561e1244d/test/e2e/alertmanager_test.go#L51-L61

@wking
Copy link
Member Author

wking commented Jul 24, 2020

So until there is a way to make provider-agnostic volume claims, e2e-operator needs to stay on AWS? Or does this affect the other jobs too?

@simonpasquier
Copy link
Contributor

simonpasquier commented Jul 24, 2020

The e2e and e2e-upgrade jobs should be platform-agnostic.

@wking
ci-operator/config/openshift/cluster-monitoring-operator: Generic e2e…
7a91f92 
…-aws -> e2e for 4.6+

Following the pattern from e7bb102
(ci-operator/config/openshift/cluster-version-operator: Generic
e2e-gcp -> e2e for 4.4+, 2020-07-09, openshift#10152), the "e2e-aws" -> "e2e"
change declares the monitoring-operator presubmits to be
platform-agnostic.

I'm leaving e2e-aws-operator alone, because it relies on AWS-specific
gp2 storage configuration [1,2].

The AWS -> GCP change takes advantage of the platform-agnosticism to
shift CI load from AWS (where we're currently pegging Boskos lease
capacity) to GCP (where we have some spare lease capacity).

Generated by manually changing ci-operator/config/... and then running:

  $ make update

[1]: openshift#10377 (comment)
[2]: https://github.com/openshift/cluster-monitoring-operator/blob/e1caabda745caba6e4784095aebd75f561e1244d/test/e2e/alertmanager_test.go#L51-L61
@wking wking force-pushed the platform-agnostic-monitoring-presubmits branch from 7fc9554 to 7a91f92 Compare July 24, 2020 14:32
@wking
Copy link
Member Author

wking commented Jul 24, 2020

Rebased on master and dropped the e2e-aws-operator changes with 7fc955436f -> 7a91f92.

@smarterclayton
Copy link
Contributor

smarterclayton commented Jul 30, 2020

/retest

@wking wking mentioned this pull request Aug 1, 2020
Merged
@lilic
Copy link
Contributor

lilic commented Aug 3, 2020

/retest

1 similar comment
@simonpasquier
Copy link
Contributor

simonpasquier commented Aug 10, 2020

/retest

@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Aug 10, 2020 •
edited
Loading

@wking: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/rehearse/openshift/cluster-monitoring-operator/release-4.7/e2e-operator 7fc955436f39cc818b5dc06e6dcb732b56eee14a link /test pj-rehearse
ci/rehearse/openshift/cluster-monitoring-operator/release-4.6/e2e-operator 7fc955436f39cc818b5dc06e6dcb732b56eee14a link /test pj-rehearse
ci/rehearse/openshift/cluster-monitoring-operator/master/e2e-operator 7fc955436f39cc818b5dc06e6dcb732b56eee14a link /test pj-rehearse

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@simonpasquier
Copy link
Contributor

simonpasquier commented Aug 11, 2020

Hmm, it seems that console, authentication and monitoring operators are failing because the API server can't handle routes.route.openshift.io requests. Is it something specific with GCP?

@wking
Copy link
Member Author

wking commented Aug 11, 2020 •
edited
Loading

Looks like 4.6 jobs passed, but 4.7 did not. I wouldn't worry about 4.7 yet; those should be identical to the 4.6 jobs at the moment. I suspect some buggy divergence in whick images are being sucked into the release, which will get straightened out before we fork 4.6 off of master. See here, showing no recently accepted 4.7 images.

@simonpasquier
Copy link
Contributor

simonpasquier commented Aug 12, 2020

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Aug 12, 2020
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Aug 12, 2020

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: simonpasquier, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 12, 2020
@openshift-bot
Copy link
Contributor

openshift-bot commented Aug 12, 2020

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit 20676b8 into openshift:master Aug 12, 2020
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Aug 12, 2020

@wking: Updated the following 9 configmaps:

  • job-config-4.7 configmap in namespace ci at cluster api.ci using the following files:
    • key openshift-cluster-monitoring-operator-release-4.7-presubmits.yaml using file ci-operator/jobs/openshift/cluster-monitoring-operator/openshift-cluster-monitoring-operator-release-4.7-presubmits.yaml
  • job-config-4.7 configmap in namespace ci at cluster app.ci using the following files:
    • key openshift-cluster-monitoring-operator-release-4.7-presubmits.yaml using file ci-operator/jobs/openshift/cluster-monitoring-operator/openshift-cluster-monitoring-operator-release-4.7-presubmits.yaml
  • job-config-master configmap in namespace ci at cluster api.ci using the following files:
    • key openshift-cluster-monitoring-operator-master-presubmits.yaml using file ci-operator/jobs/openshift/cluster-monitoring-operator/openshift-cluster-monitoring-operator-master-presubmits.yaml
  • job-config-4.6 configmap in namespace ci at cluster api.ci using the following files:
    • key openshift-cluster-monitoring-operator-release-4.6-presubmits.yaml using file ci-operator/jobs/openshift/cluster-monitoring-operator/openshift-cluster-monitoring-operator-release-4.6-presubmits.yaml
  • job-config-4.6 configmap in namespace ci at cluster app.ci using the following files:
    • key openshift-cluster-monitoring-operator-release-4.6-presubmits.yaml using file ci-operator/jobs/openshift/cluster-monitoring-operator/openshift-cluster-monitoring-operator-release-4.6-presubmits.yaml
  • job-config-master configmap in namespace ci at cluster app.ci using the following files:
    • key openshift-cluster-monitoring-operator-master-presubmits.yaml using file ci-operator/jobs/openshift/cluster-monitoring-operator/openshift-cluster-monitoring-operator-master-presubmits.yaml
  • ci-operator-master-configs configmap in namespace ci at cluster app.ci using the following files:
    • key openshift-cluster-monitoring-operator-master.yaml using file ci-operator/config/openshift/cluster-monitoring-operator/openshift-cluster-monitoring-operator-master.yaml
  • ci-operator-4.6-configs configmap in namespace ci at cluster app.ci using the following files:
    • key openshift-cluster-monitoring-operator-release-4.6.yaml using file ci-operator/config/openshift/cluster-monitoring-operator/openshift-cluster-monitoring-operator-release-4.6.yaml
  • ci-operator-4.7-configs configmap in namespace ci at cluster app.ci using the following files:
    • key openshift-cluster-monitoring-operator-release-4.7.yaml using file ci-operator/config/openshift/cluster-monitoring-operator/openshift-cluster-monitoring-operator-release-4.7.yaml
Details

In response to this:

Following the pattern from e7bb102 (#10152), the "e2e-aws" -> "e2e" change declares the monitoring-operator presubmits to be platform-agnostic.

The AWS -> GCP change takes advantage of the platform-agnosticism to shift CI load from AWS (where we're currently pegging Boskos lease capacity) to GCP (where we have some spare lease capacity).

Generated by manually changing ci-operator/config/... and then running:

$ make update

Process docs in flight with #10166.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@wking wking deleted the platform-agnostic-monitoring-presubmits branch August 12, 2020 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lilic lilic Awaiting requested review from lilic

@pgier pgier Awaiting requested review from pgier

Assignees

@simonpasquier simonpasquier

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@wking @lilic @simonpasquier @smarterclayton @openshift-ci-robot @openshift-bot @openshift-merge-robot