-
Notifications
You must be signed in to change notification settings - Fork 4k
Add BCR publishing automation with workflow and configuration #12555
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mmorel-35
wants to merge
1
commit into
grpc:master
Choose a base branch
from
mmorel-35:bcr
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,195 @@ | ||
| # Bazel Central Registry (BCR) Publishing for grpc-java | ||
|
|
||
| This document explains how grpc-java releases are published to the [Bazel Central Registry (BCR)](https://github.com/bazelbuild/bazel-central-registry). | ||
|
|
||
| ## Overview | ||
|
|
||
| The BCR is the official registry for Bazel modules, allowing users to depend on grpc-java using Bazel's module system (bzlmod). When a new version of grpc-java is released, it is automatically published to the BCR via a GitHub Actions workflow. | ||
|
|
||
| ## How It Works | ||
|
|
||
| ### Automated Publishing Workflow | ||
|
|
||
| The publishing process is automated through the GitHub Actions workflow located at `.github/workflows/publish-to-bcr.yml`. This workflow: | ||
|
|
||
| 1. **Triggers automatically** when a new release is published on GitHub | ||
| 2. **Can be manually triggered** via workflow_dispatch for re-runs or troubleshooting | ||
| 3. **Uses the official publish-to-bcr action** from bazel-contrib | ||
| 4. **Generates attestations** for release provenance and security | ||
| 5. **Creates a pull request** directly to the official Bazel Central Registry (https://github.com/bazelbuild/bazel-central-registry) with the new version | ||
|
|
||
| ### BCR Configuration Files | ||
|
|
||
| The BCR configuration is stored in the `.bcr/` directory: | ||
|
|
||
| - **`metadata.template.json`**: Contains module metadata including homepage, maintainers, and repository information | ||
| - **`source.template.json`**: Specifies how to download the source archive (uses GitHub release tags) | ||
| - **`presubmit.yml`**: Defines CI tests that run in the BCR before accepting the new version | ||
|
|
||
| ### Presubmit Testing | ||
|
|
||
| Before a new version is accepted into the BCR, it must pass presubmit tests defined in `.bcr/presubmit.yml`. These tests: | ||
|
|
||
| - **Build key targets** from the grpc-java module to ensure basic functionality | ||
| - **Test the examples module** which demonstrates using grpc-java as a dependency | ||
| - **Run on multiple platforms**: Debian, macOS, Ubuntu, Windows | ||
| - **Test with multiple Bazel versions**: 7.x and 8.x | ||
|
|
||
| The examples directory serves as a test module that uses `local_path_override` to depend on the main grpc-java module, providing comprehensive integration testing. | ||
|
|
||
| ## Setup and Configuration | ||
|
|
||
| ### For Repository Maintainers | ||
|
|
||
| To enable BCR publishing for grpc-java, follow these steps: | ||
|
|
||
| #### 1. Create a Personal Access Token (PAT) | ||
|
|
||
| A GitHub Personal Access Token is required to create pull requests to the Bazel Central Registry. | ||
|
|
||
| **Creating a Classic PAT:** | ||
|
|
||
| 1. Go to GitHub Settings → Developer settings → Personal access tokens → Tokens (classic) | ||
| 2. Click "Generate new token (classic)" | ||
| 3. Give it a descriptive name like "grpc-java BCR Publishing" | ||
| 4. Select the following scopes: | ||
| - `repo` (Full control of private repositories) | ||
| - `workflow` (Update GitHub Action workflows) | ||
| 5. Set an appropriate expiration (consider using no expiration for automation) | ||
| 6. Click "Generate token" and **save the token securely** | ||
|
|
||
| **Note about Fine-grained PATs:** | ||
| Fine-grained tokens are not fully supported yet because they cannot open pull requests against public repositories (see [GitHub roadmap issue #600](https://github.com/github/roadmap/issues/600)). | ||
|
|
||
| #### 2. Configure Repository Secrets | ||
|
|
||
| 1. Go to the grpc-java repository Settings → Secrets and variables → Actions | ||
| 2. Click "New repository secret" | ||
| 3. Name: `BCR_PUBLISH_TOKEN` | ||
| 4. Value: Paste the PAT you created | ||
| 5. Click "Add secret" | ||
|
|
||
| #### 3. Verify Workflow Permissions | ||
|
|
||
| Ensure the workflow has the necessary permissions in the repository settings: | ||
|
|
||
| - Go to Settings → Actions → General → Workflow permissions | ||
| - Ensure "Read and write permissions" is selected | ||
| - Ensure "Allow GitHub Actions to create and approve pull requests" is checked | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure if this permission is granted and I don't have access to the repository options in Settings. Adding Eric as reviewer who has that access. |
||
|
|
||
| ## Publishing a New Release | ||
|
|
||
| ### Automatic Publishing | ||
|
|
||
| When you create a new release on GitHub: | ||
|
|
||
| 1. Go to the grpc-java repository | ||
| 2. Create a new release with a tag (e.g., `v1.76.0`) | ||
| 3. The `publish-to-bcr.yml` workflow will automatically trigger | ||
| 4. Monitor the workflow run in the Actions tab | ||
| 5. If successful, a PR will be created in the BCR repository | ||
|
|
||
| ### Manual Publishing | ||
|
|
||
| If you need to manually trigger the publishing workflow: | ||
|
|
||
| 1. Go to Actions → Publish to BCR | ||
| 2. Click "Run workflow" | ||
| 3. Enter the tag name (e.g., `v1.76.0`) | ||
| 4. Click "Run workflow" | ||
|
|
||
| ### Monitoring the Workflow | ||
|
|
||
| 1. Go to the Actions tab in the grpc-java repository | ||
| 2. Look for the "Publish to BCR" workflow run | ||
| 3. Check the logs for any errors or issues | ||
| 4. If successful, you'll see a link to the created BCR pull request | ||
|
|
||
| ## BCR Pull Request Review | ||
|
|
||
| After the workflow creates a PR in the BCR: | ||
|
|
||
| 1. The PR will be automatically assigned to the module maintainers | ||
| 2. BCR CI will run presubmit tests to validate the new version | ||
| 3. Maintainers should review and approve the PR | ||
| 4. Once approved and tests pass, the BCR team will merge the PR | ||
| 5. The new version becomes available in the BCR | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Common Issues | ||
|
|
||
| **Workflow fails with authentication error:** | ||
| - Verify the `BCR_PUBLISH_TOKEN` secret is correctly set | ||
| - Ensure the PAT has not expired | ||
| - Check that the PAT has the required `repo` and `workflow` scopes | ||
|
|
||
| **Presubmit tests fail in BCR:** | ||
| - Review the test logs in the BCR PR | ||
| - Ensure the MODULE.bazel file is correctly formatted | ||
| - Verify all dependencies are available in BCR | ||
| - Test locally using the BCR's local testing instructions | ||
|
|
||
| **PR creation fails:** | ||
| - Verify the registry fork exists and is accessible | ||
| - Ensure the PAT has permissions for the fork repository | ||
| - Check that the workflow has correct permissions | ||
|
|
||
| **Version already exists:** | ||
| - BCR is add-only; you cannot replace existing versions | ||
| - If a fix is needed, it should be fixed upstream and a new version submitted | ||
| - If the fix is only in BCR patches (nothing to fix upstream), use the `.bcr.N` suffix format (e.g., `1.75.0.bcr.1`) | ||
|
|
||
| ### Getting Help | ||
|
|
||
| - Check the [BCR documentation](https://github.com/bazelbuild/bazel-central-registry/blob/main/docs/README.md) | ||
| - Review the [publish-to-bcr documentation](https://github.com/bazel-contrib/publish-to-bcr) | ||
| - Contact BCR maintainers at [email protected] | ||
| - File issues in the appropriate repository | ||
|
|
||
| ## Maintainers | ||
|
|
||
| The following people are listed as maintainers for grpc-java in the BCR and will be notified of new PRs: | ||
|
|
||
| - Eric Anderson (@ejona86) | ||
| - Abhishek Agrawal (@AgraVator) | ||
| - Kannan J (@kannanjgithub) | ||
| - MV Shiva (@shivaspeaks) | ||
|
|
||
| Maintainers have approval rights for BCR PRs related to grpc-java, even without direct write access to the BCR repository. | ||
|
|
||
| ## Patches | ||
|
|
||
| The BCR may include patches to the grpc-java module to ensure compatibility with the rest of the BCR ecosystem. These patches are typically applied by BCR maintainers and may include: | ||
|
|
||
| - Updating dependency versions to use BCR-compatible versions | ||
| - Adding missing dependencies required for Bazel module builds | ||
| - Adjusting build configurations for BCR compatibility | ||
|
|
||
| **Note:** If you need to include custom patches with your release, you can add them to the `.bcr/patches/` directory in the grpc-java repository. All patches must: | ||
| - Have the `.patch` extension | ||
| - Be in the `-p1` format (apply from the repository root) | ||
| - Be listed in the BCR entry's `source.json` file (automatically handled by the publish-to-bcr workflow) | ||
|
|
||
| The publish-to-bcr workflow will automatically include any patches found in `.bcr/patches/` when creating the BCR entry. | ||
|
|
||
| ## Module Structure | ||
|
|
||
| grpc-java is published as a single module to the BCR: | ||
|
|
||
| - **Main module**: The root `MODULE.bazel` defines the grpc-java module | ||
| - **Test module**: The `examples/` directory serves as a test module during presubmit | ||
|
|
||
| If additional modules need to be published from this repository in the future, they can be configured using the `moduleRoots` field in `.bcr/config.yml`. | ||
|
|
||
| ## References | ||
|
|
||
| - [Bazel Central Registry](https://github.com/bazelbuild/bazel-central-registry) | ||
| - [BCR Contribution Guidelines](https://github.com/bazelbuild/bazel-central-registry/blob/main/docs/README.md) | ||
| - [publish-to-bcr GitHub Action](https://github.com/bazel-contrib/publish-to-bcr) | ||
| - [Bzlmod User Guide](https://bazel.build/docs/bzlmod) | ||
| - [grpc-java in BCR](https://github.com/bazelbuild/bazel-central-registry/tree/main/modules/grpc-java) | ||
|
|
||
| ## License | ||
|
|
||
| This documentation and the BCR configuration files are part of grpc-java and are subject to the same Apache 2.0 license. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| { | ||
| "homepage": "https://github.com/grpc/grpc-java", | ||
| "maintainers": [ | ||
| { | ||
| "name": "Eric Anderson", | ||
| "email": "[email protected]", | ||
| "github": "ejona86", | ||
| "github_user_id": 2811396 | ||
| }, | ||
| { | ||
| "name": "Abhishek Agrawal", | ||
| "github": "AgraVator", | ||
| "github_user_id": 81427947 | ||
| }, | ||
| { | ||
| "name": "Kannan J", | ||
| "github": "kannanjgithub", | ||
| "github_user_id": 29600796 | ||
| }, | ||
| { | ||
| "name": "MV Shiva", | ||
| "github": "shivaspeaks", | ||
| "github_user_id": 56782215 | ||
| } | ||
| ], | ||
| "repository": [ | ||
| "github:grpc/grpc-java" | ||
| ], | ||
| "versions": [], | ||
| "yanked_versions": {} | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| # BCR presubmit configuration for grpc-java | ||
| # Tests the main module and the examples test module | ||
|
|
||
| # Test the main grpc-java module | ||
| matrix: | ||
| platform: ["debian10", "macos", "ubuntu2004", "windows"] | ||
| bazel: ["7.x", "8.x"] | ||
| tasks: | ||
| verify_targets: | ||
| platform: ${{ platform }} | ||
| bazel: ${{ bazel }} | ||
| build_targets: | ||
| - '@grpc-java//:java_grpc_library__external_repo_test' | ||
|
|
||
| # Test the examples module which uses grpc-java as a dependency | ||
| bcr_test_module: | ||
| module_path: "examples" | ||
| matrix: | ||
| platform: ["debian10", "macos", "ubuntu2004", "windows"] | ||
| bazel: ["7.x", "8.x"] | ||
| tasks: | ||
| run_test_module: | ||
| name: "Run examples test module" | ||
| platform: ${{ platform }} | ||
| bazel: ${{ bazel }} | ||
| build_targets: | ||
| - '//src/main/java/io/grpc/examples/helloworld:hello-world-client' | ||
| - '//src/main/java/io/grpc/examples/helloworld:hello-world-server' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| { | ||
| "integrity": "", | ||
| "strip_prefix": "grpc-java-{VERSION}", | ||
| "url": "https://github.com/grpc/grpc-java/archive/refs/tags/v{VERSION}.tar.gz" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| name: Publish to BCR | ||
|
|
||
| # This workflow publishes new releases to the Bazel Central Registry (BCR) | ||
| # It is triggered automatically when a new release is created | ||
|
|
||
| on: | ||
| # Automatically run when a release is published | ||
| release: | ||
| types: [published] | ||
|
|
||
| # Allow manual triggering for re-runs or troubleshooting | ||
| workflow_dispatch: | ||
| inputs: | ||
| tag_name: | ||
| description: 'Tag name to publish (e.g., v1.75.0)' | ||
| required: true | ||
| type: string | ||
|
|
||
| jobs: | ||
| publish: | ||
| name: Publish to Bazel Central Registry | ||
| # Use the reusable workflow from publish-to-bcr | ||
| uses: bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml@v6 | ||
| with: | ||
| tag_name: ${{ github.event_name == 'release' && github.event.release.tag_name || inputs.tag_name }} | ||
| # Repository to submit the PR to (using official BCR) | ||
| # You can also use a fork by specifying: your-org/bazel-central-registry | ||
| registry_fork: bazelbuild/bazel-central-registry | ||
| # Generate attestations for provenance | ||
| attest: true | ||
| # Required permissions for the workflow | ||
| permissions: | ||
| contents: write | ||
| # Required for generating attestations | ||
| id-token: write | ||
| attestations: write | ||
| secrets: | ||
| # Personal Access Token with 'repo' and 'workflow' permissions | ||
| # Must be stored as a repository secret named BCR_PUBLISH_TOKEN | ||
| publish_token: ${{ secrets.BCR_PUBLISH_TOKEN }} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have added a new repository secret with this name.