Skip to content

Conversation

@jinyunma
Copy link
Contributor

@jinyunma jinyunma commented Feb 7, 2025

  1. Panic error is thrown from getNextAvailableIP when installing in shared VPC with minimal permissions, see failed job: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_installer/9422/pull-ci-openshift-installer-main-e2e-azure-ovn-shared-vpc/1885421707863265280 (slack thread)

Add additional permission "Microsoft.Network/virtualNetworks/checkIpAddressAvailability/read" to fix the issue

  1. Read install-config and assign proper permission list based on various config, remove env ENABLE_MIN_PERMISSION_FOR_MARKETPLACE and ENABLE_MIN_PERMISSION_FOR_DES.
  2. Extract custom role creation from step "azure-provision-service-principal-minimal-permission" as a separated step "azure-provision-custom-role", to cover case "install cluster with managed identity assigned minimal permissions" which only needs to custom role.
    And introduce new chain "azure-provision-service-principal-minimal-permission" to include both steps.
  3. Extract role assignment from step azure-provision-disk-encryption-set, azure-provision-vnet,ipi-conf-azure-resourcegroup as separated steps, only run role assignment steps in dedicated jobs instead of using ENV to control it.

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 7, 2025
@openshift-ci openshift-ci bot requested review from dgoodwin and sosiouxme February 7, 2025 04:25
@jinyunma
Copy link
Contributor Author

jinyunma commented Feb 7, 2025

/pj-rehearse pull-ci-openshift-installer-main-e2e-azure-ovn-shared-vpc

@openshift-ci-robot
Copy link
Contributor

@jinyunma: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@jinyunma
Copy link
Contributor Author

jinyunma commented Feb 7, 2025

Installation on shared vpc succeeded in job

/pj-rehearse ack

cc @jianlinliu @gpei to review

@openshift-ci-robot
Copy link
Contributor

@jinyunma: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci-robot openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Feb 7, 2025

# optional permissions when installing cluster in existing vnet
required_permissions="""
\"Microsoft.Network/virtualNetworks/checkIpAddressAvailability/read\",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The permission is needed by recent PR change, but not for all the previous OCP versions, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be related with bug https://issues.redhat.com//browse/OCPBUGS-37442, which has been backported to 4.17. So this permission is required on 4.17+ CAPI install.
Will file doc bug to request to add the permission on 4.17+.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we place the needed permission into the if block on line 281?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error happens when creating manifests, this should impact both IPI and UPI.
But I can update to only apply permission on 4.17+.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jinyunma one more question, does it only affect the installation for existing vnet? If so, do we need a separate env var to control it, so we can skip this permission for IPI install without existing vnet?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion.
As discussed offline, will read install-config and assign proper permissions based on various config.

@jinyunma jinyunma force-pushed the az-byo-vpc-permission branch from 11e52b0 to 69a16bb Compare February 7, 2025 08:05
@jianlinliu
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Feb 7, 2025
@openshift-ci-robot openshift-ci-robot removed the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Feb 7, 2025
@jinyunma
Copy link
Contributor Author

jinyunma commented Feb 7, 2025

/pj-rehearse ack

@openshift-ci-robot
Copy link
Contributor

@jinyunma: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci-robot openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Feb 7, 2025
@jinyunma
Copy link
Contributor Author

jinyunma commented Feb 7, 2025

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 7, 2025
@jinyunma jinyunma force-pushed the az-byo-vpc-permission branch from 69a16bb to d85e505 Compare February 7, 2025 11:13
@openshift-ci openshift-ci bot removed lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Feb 7, 2025
@openshift-ci-robot openshift-ci-robot removed the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Feb 7, 2025
@jinyunma jinyunma force-pushed the az-byo-vpc-permission branch from d85e505 to 70960ab Compare February 7, 2025 11:24
@jinyunma
Copy link
Contributor Author

jinyunma commented Feb 7, 2025

/pj-rehearse pull-ci-openshift-installer-main-azure-ovn-marketplace-images periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-azure-ipi-des-mini-perm-fips-amd-f28-destructive periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-azure-upi-mini-perm-amd-f28-destructive periodic-ci-openshift-openshift-tests-private-release-4.18-amd64-nightly-azure-ipi-marketplace-mini-perm-f7

@openshift-ci-robot
Copy link
Contributor

@jinyunma: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@jinyunma jinyunma force-pushed the az-byo-vpc-permission branch 2 times, most recently from b32ff88 to 34d925a Compare February 7, 2025 14:58
@jinyunma
Copy link
Contributor Author

jinyunma commented Feb 7, 2025

/pj-rehearse pull-ci-openshift-installer-main-e2e-azure-ovn-shared-vpc periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-azure-ipi-des-mini-perm-fips-amd-f28-destructive periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-azure-upi-mini-perm-amd-f28-destructive periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-azure-ipi-oidc-mini-perm-arm-f7

@patrickdillon
Copy link
Contributor

/approve

1 similar comment
@patrickdillon
Copy link
Contributor

/approve

@jianlinliu
Copy link
Contributor

cc @liangxia to approve

@jianlinliu
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Feb 25, 2025
@jinyunma
Copy link
Contributor Author

/pj-rehearse ack

@openshift-ci-robot
Copy link
Contributor

@jinyunma: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci-robot openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Feb 25, 2025
- ref: azure-provision-service-principal-minimal-permission
- ref: ipi-conf
- ref: ipi-conf-azure-default
- chain: azure-provision-service-principal-minimal-permission
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to change openshift-priv ones ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are those openshift-priv jobs automatically synced up by some way? I searched them when updating this chain.
I'm not sure when this job will be triggered. Once triggered, the change is needed here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we updated the openshift-priv ones on purpose because of mini-perm change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, those openshift-priv are used by ART team. Generally we don't touch them as there are auto sync mechanism.
If you are sure this is exactly needed, I can approve it. Just let me know.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, the change is required regardless in which way to update them.
Thanks for approving this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@liangxia I removed the change in openshift-priv job config, PTAL.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems that the change must be done manually in this PR for openshift-priv jobs, otherwise job pull-ci-openshift-release-master-ci-operator-config failed, because some envs are removed on purpose in this PR.

So I changed back for openshift-priv jobs.

@jinyunma jinyunma force-pushed the az-byo-vpc-permission branch from ae630e2 to 4ac0a99 Compare February 25, 2025 05:21
@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Feb 25, 2025
@jinyunma jinyunma force-pushed the az-byo-vpc-permission branch from 4ac0a99 to c98d27d Compare February 25, 2025 05:23
@openshift-ci-robot openshift-ci-robot removed the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Feb 25, 2025
@liangxia
Copy link
Member

/approve

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 25, 2025
@jinyunma jinyunma force-pushed the az-byo-vpc-permission branch from c98d27d to e0ba71a Compare February 25, 2025 05:38
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 25, 2025

@jinyunma: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/rehearse/periodic-ci-openshift-openshift-tests-private-release-4.18-amd64-nightly-azure-ipi-marketplace-mini-perm-f7 70960ab link unknown /pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.18-amd64-nightly-azure-ipi-marketplace-mini-perm-f7
ci/rehearse/periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-azure-ipi-oidc-mini-perm-amd-f28-destructive f6eaab0 link unknown /pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-azure-ipi-oidc-mini-perm-amd-f28-destructive
ci/rehearse/periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-azure-ipi-des-mini-perm-arm-f14 eff43be link unknown /pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-azure-ipi-des-mini-perm-arm-f14
ci/rehearse/periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-azure-ipi-private-spec-net-type-spec-perm-arm-f7 deb286f link unknown /pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.18-multi-nightly-azure-ipi-private-spec-net-type-spec-perm-arm-f7

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-ci-robot
Copy link
Contributor

[REHEARSALNOTIFIER]
@jinyunma: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-external-dns-operator-main-e2e-azure-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-main-e2e-azure-infoblox-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-4.15-e2e-azure-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-1.2-e2e-azure-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-1.2-e2e-azure-infoblox-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-1.0-e2e-azure-ovn-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-1.0-e2e-azure-ovn-infoblox-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-0.1-e2e-azure-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-1.1-e2e-azure-ovn-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-1.1-e2e-azure-ovn-infoblox-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-4.17-e2e-azure-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-4.17-e2e-azure-infoblox-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-4.16-e2e-azure-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-external-dns-operator-release-4.16-e2e-azure-infoblox-operator openshift/external-dns-operator presubmit Registry content changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-azure-4.15-nightly-x86-data-path-9nodes openshift-qe/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-azure-4.16-nightly-x86-data-path-9nodes openshift-qe/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-azure-4.19-nightly-x86-data-path-9nodes openshift-qe/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-qe-ocp-qe-perfscale-ci-main-azure-4.16-nightly-arm-control-plane-3nodes-arm openshift-qe/ocp-qe-perfscale-ci presubmit Registry content changed
pull-ci-openshift-sandboxed-containers-operator-release-1.8-sandboxed-containers-operator-e2e openshift/sandboxed-containers-operator presubmit Registry content changed
pull-ci-openshift-sandboxed-containers-operator-devel-sandboxed-containers-operator-e2e openshift/sandboxed-containers-operator presubmit Registry content changed
pull-ci-openshift-cluster-api-master-e2e-azure-capi-techpreview openshift/cluster-api presubmit Registry content changed
pull-ci-openshift-cluster-api-release-4.20-e2e-azure-capi-techpreview openshift/cluster-api presubmit Registry content changed
pull-ci-openshift-cluster-api-release-4.19-e2e-azure-capi-techpreview openshift/cluster-api presubmit Registry content changed
pull-ci-openshift-cluster-api-release-4.18-e2e-azure-capi-techpreview openshift/cluster-api presubmit Registry content changed
pull-ci-openshift-cluster-config-operator-master-e2e-azure openshift/cluster-config-operator presubmit Registry content changed

A total of 2467 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@gpei
Copy link
Contributor

gpei commented Feb 25, 2025

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Feb 25, 2025
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 25, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gpei, jianlinliu, jinyunma, liangxia, patrickdillon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jinyunma
Copy link
Contributor Author

/pj-rehearse ack

@openshift-ci-robot
Copy link
Contributor

@jinyunma: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci-robot openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Feb 25, 2025
@openshift-merge-bot openshift-merge-bot bot merged commit 10c94a5 into openshift:master Feb 25, 2025
15 checks passed
@jinyunma jinyunma deleted the az-byo-vpc-permission branch February 25, 2025 06:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. rehearsals-ack Signifies that rehearsal jobs have been acknowledged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants