Skip to content

WIP: sync: Distinguish between reconcile, initial, and sync states #131

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
smarterclayton wants to merge 5 commits into from
Closed

WIP: sync: Distinguish between reconcile, initial, and sync states #131

smarterclayton wants to merge 5 commits into from

Conversation

@smarterclayton
Copy link
Contributor

@smarterclayton smarterclayton commented Mar 8, 2019

Identify what sort of change we are making to the cluster in order
to better handle the different states.

TODO:

  • Sync differently depending on state

@openshift-ci-robot openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 8, 2019
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Mar 8, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: smarterclayton

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 8, 2019
@smarterclayton
sync: Distinguish between reconcile, initial, and sync states
8812c9b 
Identify what sort of change we are making to the cluster in order
to better handle the different states.
@smarterclayton
sync: React differently in sync when in different modes
5ec83d9 
When initializing, don't wait for Failing=false in cluster operators.
When reconciling, only error out if the operator is unavailable.
When upgrading, have roughly the same logic as before (be cautious).
@smarterclayton
sync: Double max parallelism of sync worker
f442641 
The 0000_70 run level has ~13 operators. It appears we can afford
to go a bit broader (although the ideal solution is to pause a task
group when we hit a CO, flush out the others, then wait).
@smarterclayton
Copy link
Contributor Author

smarterclayton commented Mar 8, 2019

/test images

@smarterclayton
Copy link
Contributor Author

smarterclayton commented Mar 9, 2019 •
edited
Loading

With this PR and this payload ordering the payload applied in 10m, down from about 20. Console was the limiting reagent, followed by monitoring.

0000_05_config-operator
0000_10_cloud-credential-operator
0000_10_cluster-dns-operator
0000_10_cluster-network-operator
0000_10_kube-apiserver-operator
0000_10_machine-api-operator
0000_10_machine-approver
0000_10_service-ca-operator
0000_30_cluster-ingress-operator
0000_30_kube-controller-manager-operator
0000_30_kube-scheduler-operator
0000_30_machine-config-operator
0000_30_olm
0000_30_openshift-apiserver-operator
0000_30_openshift-controller-manager-operator
0000_70_cluster-authentication-operator
0000_70_cluster-autoscaler-operator
0000_70_cluster-image-registry-operator
0000_70_cluster-monitoring-operator
0000_70_cluster-node-tuning-operator
0000_70_cluster-samples-operator
0000_70_cluster-storage-operator
0000_70_cluster-svcat-apiserver-operator
0000_70_cluster-svcat-controller-manager-operator
0000_70_console-operator
0000_70_operator-marketplace

@smarterclayton
Copy link
Contributor Author

smarterclayton commented Mar 9, 2019

/retest

@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Mar 9, 2019

@smarterclayton: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
ci/prow/e2e-aws 605699c link /test e2e-aws

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

This was referenced Mar 9, 2019
Merged
Merged
@smarterclayton
Copy link
Contributor Author

smarterclayton commented Mar 9, 2019

/hold

Going to split this up into multiple independent PRs now that it has proved its worth.

@openshift-ci-robot openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 9, 2019
@smarterclayton smarterclayton mentioned this pull request Mar 9, 2019
Closed
@smarterclayton
Copy link
Contributor Author

smarterclayton commented Mar 13, 2019

Replaced with #136

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@abhinavdahiya abhinavdahiya Awaiting requested review from abhinavdahiya

@wking wking Awaiting requested review from wking

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@smarterclayton @openshift-ci-robot