Skip to content

Ensure quorum restore test follows the documented procedure #25774

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
smarterclayton merged 2 commits into from
Jun 2, 2021
Merged

Ensure quorum restore test follows the documented procedure #25774

smarterclayton merged 2 commits into from
Jun 2, 2021

Conversation

@marun
Copy link
Contributor

@marun marun commented Dec 17, 2020 •
edited
Loading

The quorum restore test was previously deleting machines and then attempting to take a backup such that the backup script would race with machine removal. The backup script only supports taking a backup of a cluster that still has quorum. Taking the backup before machine deletion removes the race condition.

This PR also updates the test to follow the documented procedure after executing the restore script. It reuses steps defined for the backup-restore test added in #25723.

/cc @hexfusion @smarterclayton

@marun marun mentioned this pull request Dec 17, 2020
Closed
@marun
Copy link
Contributor Author

marun commented Dec 17, 2020

/test e2e-gcp-disruptive
/test e2e-aws-disruptive

@marun
Copy link
Contributor Author

marun commented Dec 17, 2020

/test e2e-gcp

Not ideal that this isn't passing given that there is no way this PR affects it.

@marun
Copy link
Contributor Author

marun commented Dec 17, 2020

/test e2e-aws-disruptive

@openshift-merge-robot
Copy link
Contributor

openshift-merge-robot commented Dec 17, 2020

@marun: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-agnostic-cmd 353caec link /test e2e-agnostic-cmd
ci/prow/e2e-gcp-disruptive 353caec link /test e2e-gcp-disruptive
ci/prow/e2e-gcp 353caec link /test e2e-gcp
ci/prow/e2e-aws-disruptive 353caec link /test e2e-aws-disruptive

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@marun
Copy link
Contributor Author

marun commented Dec 18, 2020

/test e2e-gcp
/test e2ed-aws-disruptive

@marun
Copy link
Contributor Author

marun commented Dec 18, 2020

/test e2e-aws-disruptive

@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 18, 2020

@marun: The specified target(s) for /test were not found.
The following commands are available to trigger jobs:

  • /test e2e-agnostic-cmd
  • /test e2e-aws
  • /test e2e-aws-csi
  • /test e2e-aws-disruptive
  • /test e2e-aws-fips
  • /test e2e-aws-image-registry
  • /test e2e-aws-jenkins
  • /test e2e-aws-multitenant
  • /test e2e-aws-ovn
  • /test e2e-aws-proxy
  • /test e2e-aws-serial
  • /test e2e-aws-upgrade
  • /test e2e-azure
  • /test e2e-gcp
  • /test e2e-gcp-builds
  • /test e2e-gcp-csi
  • /test e2e-gcp-disruptive
  • /test e2e-gcp-image-ecosystem
  • /test e2e-gcp-upgrade
  • /test e2e-openstack
  • /test e2e-vsphere
  • /test images
  • /test verify
  • /test verify-deps
  • /test extended_gssapi
  • /test extended_ldap_groups
  • /test extended_networking

Use /test all to run the following jobs:

  • pull-ci-openshift-origin-master-e2e-agnostic-cmd
  • pull-ci-openshift-origin-master-e2e-aws-csi
  • pull-ci-openshift-origin-master-e2e-aws-fips
  • pull-ci-openshift-origin-master-e2e-aws-serial
  • pull-ci-openshift-origin-master-e2e-gcp
  • pull-ci-openshift-origin-master-e2e-gcp-csi
  • pull-ci-openshift-origin-master-e2e-gcp-upgrade
  • pull-ci-openshift-origin-master-images
  • pull-ci-openshift-origin-master-verify
  • pull-ci-openshift-origin-master-verify-deps
Details

In response to this:

/test gcp

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@marun
Copy link
Contributor Author

marun commented Dec 18, 2020

/test e2e-gcp

@marun
Copy link
Contributor Author

marun commented Jan 4, 2021

/retest

@marun
Copy link
Contributor Author

marun commented Jan 4, 2021

/test e2e-aws-disruptive
/test e2e-gcp-disruptive

@marun
Copy link
Contributor Author

marun commented Jan 5, 2021

/test e2e-aws-disruptive

@marun
Copy link
Contributor Author

marun commented Jan 5, 2021

/test e2e-aws-disruptive
/test e2e-gcp-disruptive

1 similar comment
@marun
Copy link
Contributor Author

marun commented Jan 5, 2021

/test e2e-aws-disruptive
/test e2e-gcp-disruptive

@marun
Copy link
Contributor Author

marun commented Jan 5, 2021

/retest

@marun
Copy link
Contributor Author

marun commented Jan 6, 2021

/test e2e-aws-disruptive
/test e2e-gcp-disruptive

1 similar comment
@marun
Copy link
Contributor Author

marun commented Jan 6, 2021

/test e2e-aws-disruptive
/test e2e-gcp-disruptive

@marun
Copy link
Contributor Author

marun commented Jan 8, 2021

/retest

1 similar comment
@marun
Copy link
Contributor Author

marun commented Jan 12, 2021

/retest

@marun marun changed the title Fix quorum restore DR test to take backup before breaking quorum Bug 1886160: Fix quorum restore DR test to take backup before breaking quorum Jan 18, 2021
@hexfusion
Copy link
Contributor

hexfusion commented Jan 18, 2021

/lgtm

This is a step forward we can continue to iterate.

@openshift-ci-robot openshift-ci-robot added the bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. label Jan 18, 2021
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jan 18, 2021

@marun: This pull request references Bugzilla bug 1886160, which is invalid:

  • expected the bug to be in one of the following states: NEW, ASSIGNED, ON_DEV, POST, POST, but it is VERIFIED instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

Bug 1886160: Fix quorum restore DR test to take backup before breaking quorum

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Jan 18, 2021
@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 18, 2021
@marun
Copy link
Contributor Author

marun commented Jan 18, 2021

/bugzilla refresh

@openshift-ci-robot openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Jan 18, 2021
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 10, 2021
@marun
Copy link
Contributor Author

marun commented May 12, 2021

/retest

13 similar comments
@marun
Copy link
Contributor Author

marun commented May 12, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 13, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 13, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 13, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 17, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 17, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 18, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 20, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 24, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 25, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 25, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 25, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 26, 2021

/retest

@marun
Copy link
Contributor Author

marun commented May 26, 2021

/retest

openshift/cluster-machine-approver#121 disables caching, retesting to see the approver behaves correctly now.

@marun
Copy link
Contributor Author

marun commented May 27, 2021

/retest

4 similar comments
@marun
Copy link
Contributor Author

marun commented May 27, 2021

/retest

@marun
Copy link
Contributor Author

marun commented Jun 1, 2021

/retest

@marun
Copy link
Contributor Author

marun commented Jun 2, 2021

/retest

@marun
Copy link
Contributor Author

marun commented Jun 2, 2021

/retest

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 2, 2021

@marun: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-gcp-disruptive 2f82ad2 link /test e2e-gcp-disruptive
ci/prow/e2e-aws-disruptive 2f82ad2 link /test e2e-aws-disruptive

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@smarterclayton
Copy link
Contributor

smarterclayton commented Jun 2, 2021

/lgtm

Going to merge this to make it easier to get the root CMA issue fixed

@smarterclayton smarterclayton added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Jun 2, 2021
@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jun 2, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 2, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hexfusion, marun, smarterclayton

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 2, 2021
@smarterclayton smarterclayton merged commit 03a44ce into openshift:master Jun 2, 2021
@yuvalk yuvalk mentioned this pull request Apr 5, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hexfusion hexfusion Awaiting requested review from hexfusion

1 more reviewer

@smarterclayton smarterclayton smarterclayton left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@smarterclayton smarterclayton

@hexfusion hexfusion

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged. vendor-update Touching vendor dir or related files

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@marun @openshift-merge-robot @openshift-ci-robot @hexfusion @vrutkovs @smarterclayton