[feat] Split Playwright tests into 5 shards for parallel execution #5300
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Split Playwright tests into 5 shards to reduce CI runtime from 17min to ~3-4min - Each browser now runs tests in 5 parallel jobs (20 total jobs: 4 browsers × 5 shards) - Updated test-ui.yaml workflow to use matrix strategy with sharding - Added merge-reports job to consolidate test results - Updated pr-playwright-deploy.yaml to handle sharded test reports - Merge sharded reports before deploying to Cloudflare Pages This change will significantly improve CI/CD performance by running tests in parallel, matching the runtime of other workflows (~3-4 minutes). 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
🎭 Playwright Test Results✅ All tests passed across all browsers! ⏰ Completed at: 09/03/2025, 04:53:05 PM UTC 📊 Test Reports by Browser🎉 Your tests are passing across all browsers! |
- Implement weighted test distribution algorithm to balance shard execution times - Create automated shard optimization script that analyzes test complexity - Remove unnecessary sharding for fast test suites (mobile-chrome, chromium-0.5x, chromium-2x) - Update GitHub workflow to use optimized shard configuration - Add comprehensive sharding documentation The previous naive sharding caused shard 5 to take 9 minutes while others completed in 2-6 minutes. This was due to interaction.spec.ts containing 61 tests with 81 screenshot comparisons. New approach uses weighted distribution based on: - Number of tests per file - Screenshot comparison count - Test complexity and historical execution time Results: - Achieved ~4.5% imbalance (vs previous ~80%) - All chromium shards now complete in 3-4 minutes - Total CI time reduced from 9 minutes to ~4 minutes 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Member
Author
|
close as prefer simpler solution: Playwright Test Sharding by Myestery · Pull Request #5311 · Comfy-Org/ComfyUI_frontend |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Changes
test-ui.yaml
shardmatrix parameter (1-5) to run tests in parallel--shard=${{ matrix.shard }}/5flagmerge-reportsjob to consolidate test artifactspr-playwright-deploy.yaml
Benefits
Test Plan
🤖 Generated with Claude Code
┆Issue is synchronized with this Notion page by Unito