Skip to content

Conversation

@youknowriad
Copy link
Contributor

What

Reduces the unit test matrix for pull requests to run only a single Node.js version (20) and single PHP version (8.3), while maintaining full multi-version testing on trunk and release branches.

Why

  • Faster PR feedback: Reduces PR test jobs from ~32 to ~6-10, significantly improving CI turnaround time
  • Cost savings: Reduces compute resource usage on PRs where full version coverage is less critical
  • Maintains quality: Full version coverage still runs on trunk/releases where it matters most for production deployments

How

Uses conditional fromJSON() expressions in GitHub Actions matrix configuration based on github.event_name:

  • PRs: Node 20 only (5 jobs) + PHP 8.3 single-site only (1 job)
  • Trunk/releases: All versions - Node 20/22/24 (15 jobs) + PHP 7.2-8.3 with multisite variants (17 jobs)

@youknowriad youknowriad self-assigned this Oct 20, 2025
@youknowriad youknowriad added the [Type] Build Tooling Issues or PRs related to build tooling label Oct 20, 2025
@youknowriad youknowriad requested a review from desrosj as a code owner October 20, 2025 22:17
@github-actions
Copy link

github-actions bot commented Oct 20, 2025

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: youknowriad <[email protected]>
Co-authored-by: desrosj <[email protected]>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@github-actions
Copy link

Flaky tests detected in e877574.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/18667482078
📝 Reported issues:

Copy link
Member

@desrosj desrosj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of conditionally loading the matrix values (which is pretty messy and hard to read), what if there was an event key in the matrix that contains github.event_name. Then we can just have exclude statements for macos-15 & pull_request, windows-2025 & pull_request, and then any additional versions that we'd like to exclude.

Alternatively, an if condition that only runs for certain values when the triggering event is a pull request.

We should be intentional about how we limit combinations. While having fewer jobs is better for feedback, we shouldn't just trim for the sake of it. The idea behind having multiple test combinations is that problems are caught before they are merged. Before we just take a hatchet to the matrices for each job, I think we should look at how we can make each workflow more efficient first.

I also think that for PHP, we should at a minimum test the oldest and newest versions of each major PHP version (ie. 7.2 & 7.4, 8.0 & 8.4). Testing just 8.3 is opening us up merging compatibility issues that could be fixed prior to merging.

@youknowriad
Copy link
Contributor Author

We should be intentional about how we limit combinations. While having fewer jobs is better for feedback, we shouldn't just trim for the sake of it. The idea behind having multiple test combinations is that problems are caught before they are merged. Before we just take a hatchet to the matrices for each job, I think we should look at how we can make each workflow more efficient first.
I also think that for PHP, we should at a minimum test the oldest and newest versions of each major PHP version (ie. 7.2 & 7.4, 8.0 & 8.4). Testing just 8.3 is opening us up merging compatibility issues that could be fixed prior to merging.

In all the years working on Gutenberg, I don't recall a single time where I saw a Node.JS or PHP version fail but not others. I think it's pretty telling. I think we should be more aggressive here.

I'm not saying that this will never happen (a version failing but not others), but for me if it ever happens and trunk is broken for a few minutes (the time it takes to resolve or revert), I think it's totally ok and better than running multiple versions on PRs for something that has a very small chance of happening.

Copy link
Member

@ellatrix ellatrix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do this, we can discuss this further after Beta 1 and at least this will unblock a lot of PRs. In my opinion it's also much better for new contributors. We can do quick reverts and worry about PHP version numbers as committers.

Copy link
Contributor

@priethor priethor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed with @ellatrix , let's merge these changes to unblock Beta1 and refine after it if we need to add some specific versions. And thanks for caring about the CI

@youknowriad youknowriad merged commit 7b1a5c2 into trunk Oct 21, 2025
28 of 36 checks passed
@youknowriad youknowriad deleted the limit/ci-hjobs branch October 21, 2025 11:00
@youknowriad
Copy link
Contributor Author

@desrosj I had to merge this PR to improve the situation but we can iterate on this on a calmer schedule haha

@github-actions github-actions bot added this to the Gutenberg 22.0 milestone Oct 21, 2025
@priethor priethor added Backport to WP 6.9 Beta/RC Pull request that needs to be backported to the WordPress major release that's currently in beta Backport to Gutenberg RC Pull request that needs to be backported to a Gutenberg release candidate (RC) labels Oct 21, 2025
@priethor
Copy link
Contributor

I just cherry-picked this PR to the wp/latest branch to get it included in the next release: 64811f9

@priethor priethor removed the Backport to WP 6.9 Beta/RC Pull request that needs to be backported to the WordPress major release that's currently in beta label Oct 21, 2025
@priethor
Copy link
Contributor

I just cherry-picked this PR to the release/21.9 branch to get it included in the next release: 00c8c28

@priethor priethor removed the Backport to Gutenberg RC Pull request that needs to be backported to a Gutenberg release candidate (RC) label Oct 21, 2025
desrosj added a commit that referenced this pull request Nov 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Type] Build Tooling Issues or PRs related to build tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants