[bugfix] Fix E2E test report generation for non-chromium browsers #7193
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixed an issue where Playwright HTML reports were not being generated for chromium-2x, chromium-0.5x, and mobile-chrome test runs, causing 404 errors when accessing report links in PR comments. The problem was that these browsers run very few tests (1-6 tests each), and when using --reporter=html --reporter=json together, Playwright would only generate the JSON report without the HTML assets. Changed the workflow to use the blob reporter followed by merge-reports (matching the approach used for sharded chromium tests), which ensures both HTML and JSON reports are always generated with complete assets. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Contributor
📝 WalkthroughWalkthroughThe CI workflow for E2E tests has been restructured to decouple Playwright blob collection from final report generation. Tests now run with blob reporting in a dedicated step, followed by a separate step that generates HTML and JSON reports by merging the blob data. Changes
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: ASSERTIVE Plan: Pro 📒 Files selected for processing (1)
🧰 Additional context used🧠 Learnings (6)📓 Common learnings📚 Learning: 2025-11-24T19:47:22.909ZApplied to files:
📚 Learning: 2025-11-24T19:47:22.909ZApplied to files:
📚 Learning: 2025-11-24T19:48:23.088ZApplied to files:
📚 Learning: 2025-11-24T19:47:22.909ZApplied to files:
📚 Learning: 2025-11-24T19:48:23.088ZApplied to files:
🔇 Additional comments (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🎭 Playwright Test Results⏰ Completed at: 12/05/2025, 09:24:45 PM UTC 📈 Summary
📊 Test Reports by Browser
🎉 Click on the links above to view detailed test results for each browser configuration. |
🎨 Storybook Build Status✅ Build completed successfully! ⏰ Completed at: 12/05/2025, 09:15:53 PM UTC 🔗 Links🎉 Your Storybook is ready for review! |
Bundle Size ReportSummary
Category Glance Per-category breakdownApp Entry Points — 3.2 MB (baseline 3.2 MB) • ⚪ 0 BMain entry bundles and manifests
Graph Workspace — 975 kB (baseline 975 kB) • ⚪ 0 BGraph editor runtime, canvas, workflow orchestration
Views & Navigation — 6.54 kB (baseline 6.54 kB) • ⚪ 0 BTop-level views, pages, and routed surfaces
Panels & Settings — 298 kB (baseline 298 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens
UI Components — 173 kB (baseline 173 kB) • ⚪ 0 BReusable component library chunks
Data & Services — 12.5 kB (baseline 12.5 kB) • ⚪ 0 BStores, services, APIs, and repositories
Utilities & Hooks — 2.94 kB (baseline 2.94 kB) • ⚪ 0 BHelpers, composables, and utility bundles
Vendor & Third-Party — 8.56 MB (baseline 8.56 MB) • ⚪ 0 BExternal libraries and shared vendor chunks
Other — 3.81 MB (baseline 3.81 MB) • ⚪ 0 BBundles that do not match a named category
|
dosubot
bot
added
the
size:S
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a bug where Playwright HTML reports were not being generated for non-chromium browsers (chromium-2x, chromium-0.5x, and mobile-chrome), causing 404 errors when accessing report links. The root cause was that these browsers run very few tests, and when using multiple reporters simultaneously, Playwright would only generate the JSON report without HTML assets.
Key Changes:
- Switched non-chromium browser tests to use blob reporter first, then merge-reports for generating HTML and JSON
- This aligns the approach with the existing chromium sharded test pattern, ensuring consistent report generation regardless of test count
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes an issue where Playwright HTML reports were not being generated for
chromium-2x,chromium-0.5x, andmobile-chrometest runs, causing 404 errors when accessing report links in PR comments.Problem
Only the first report link (chromium) had contents - the other three browsers returned 404 errors. Investigation revealed that artifacts for non-chromium browsers were only uploading 1 file (report.json) instead of the full HTML report with ~32 files.
The root cause was that these browsers run very few tests (1-6 tests each), and when using
--reporter=html --reporter=jsontogether, Playwright would only generate the JSON report without the HTML assets.Solution
Changed the workflow for non-chromium browsers to use the same approach as the sharded chromium tests:
--reporter=blobmerge-reportsThis ensures both HTML and JSON reports are always generated with complete assets, regardless of test count.
Testing
The fix can be verified by:
Related to #7186
🤖 Generated with Claude Code
┆Issue is synchronized with this Notion page by Unito