Playwright Test Sharding #5311
Playwright Test Sharding #5311
Conversation
🎭 Playwright Test Results✅ All tests passed across all browsers! ⏰ Completed at: 09/03/2025, 03:20:59 PM UTC 📊 Test Reports by Browser🎉 Your tests are passing across all browsers! |
Myestery
force-pushed
the
playwright-sharding
branch
from
c2fdbd2 to
52453ad
Compare
dosubot
bot
added
the
size:L
DrJKL
left a comment
DrJKL
left a comment
There was a problem hiding this comment.
🦔 I love it. I hate waiting for these to finish so much.
| - name: Cache Playwright browsers | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: ~/.cache/ms-playwright | ||
| key: playwright-browsers-${{ runner.os }}-${{ hashFiles('ComfyUI_frontend/pnpm-lock.yaml') }}-chromium | ||
| restore-keys: | | ||
| playwright-browsers-${{ runner.os }}-${{ hashFiles('ComfyUI_frontend/pnpm-lock.yaml') }}- | ||
| playwright-browsers-${{ runner.os }}- |
There was a problem hiding this comment.
The official recommendation is to not cache the browsers: https://playwright.dev/docs/ci#caching-browsers
|
LGTM, love simple solution |
|
I will merge now so I can time it with BP changes. Please address the review comments in a followup where necessary. |
|
|
||
| playwright-tests: | ||
| # Sharded chromium tests | ||
| playwright-tests-chromium-sharded: |
There was a problem hiding this comment.
https://playwright.dev/docs/test-sharding#github-actions-example
We should add a timeout for the upper limit of test run time. Hitting the timeout can be a signal that we need to increase shard totals or rethink our browser test suite.
There was a problem hiding this comment.
we should also add a comment explaining this if we do it.
* [feat] Enhance Playwright testing workflow with sharding and report merging * feat: shard playwright tests into 8 processes
This pull request updates the UI test workflow to improve scalability and reporting for Playwright tests. The main changes introduce sharding for Chromium-based tests, optimize dependency installation and caching, and add a new job to merge test reports for easier review.
Test sharding and reporting improvements:
playwright-tests-chromium-shardedto run Chromium Playwright tests in parallel shards, increasing test speed and scalability. The matrix now usesshardIndexandshardTotalinstead of individual browser variants.merge-reportsjob that collects and merges the results from all Chromium shards into a single HTML report artifact for easier review.Dependency installation and caching enhancements:
wait-for-it), and to start the ComfyUI server before running tests, ensuring the test environment is fully prepared.Workflow maintenance:
Changes
Review Focus
Screenshots (if applicable)
┆Issue is synchronized with this Notion page by Unito