[ci] Chromatic comment should be safe to fail

[snomiao]

• Chromatic comment should be safe to fail

Originally posted by @snomiao in #5046 (comment)

┆Issue is synchronized with this Notion page by Unito

[snomiao]

Let me review why CICD fails in #5046 and check chromatic.yaml and enable continue on error when find/update issue comment fails.

Also playwright comment.

---

Investigation Steps:

1. ✅ Analyze the codebase to understand the issue/question
2. ✅ Research relevant documentation and code patterns
3. ✅ Identify the root cause or provide comprehensive answer
4. ✅ Prepare a detailed investigation report

Status: Investigation COMPLETE ✅

---

📋 Investigation Report

🔍 Root Cause Analysis

Primary Issue: Both Chromatic and Playwright workflows fail at comment steps due to GitHub token permission restrictions.

Error Details:

• Error Code: 403 - "Resource not accessible by integration"
• Failed Operations: Creating/updating PR comments via GitHub API
• Affected Workflows: chromatic.yaml, test-ui.yaml

🔧 Technical Analysis

Permission Mismatch:

• YAML Configuration: Workflows specify pull-requests: write and issues: write
• Actual Token Permissions: Only read access granted (Issues: read, PullRequests: read)
• Repository Setting: Default GITHUB_TOKEN permissions restricted to read-only

Workflow Dependencies:

• Chromatic: Fails early at comment step, skipping Storybook build
• Playwright: Setup fails at comment, preventing test execution
• Both: Use edumserrano/find-create-or-update-comment@v3 action

💡 Solutions

Option 1: Enable continue-on-error (Recommended)

Option 2: Repository Settings Fix

• Go to Repository Settings → Actions → General
• Under "Workflow permissions" select "Read and write permissions"
• This requires admin access

Option 3: Conditional Comments

📋 Implementation Checklist

• Add continue-on-error: true to all comment steps in chromatic.yaml
• Add continue-on-error: true to all comment steps in test-ui.yaml
• Test PR workflow execution
• Verify core functionality (Chromatic builds, Playwright tests) works without comments
• Optional: Fix repository permissions for full commenting functionality

Impact: Comment failures will no longer block essential CI processes (visual testing, browser testing)

[snomiao]

🔍 Investigation Started

I've started investigating your question/issue. Agent 161 is analyzing the codebase and will provide a detailed report soon.

• Agent ID: 161
• Status: Investigating
• Branch: issue-5149-ci-chromatic-comment-should-be-safe-to-fail

I'll update this comment with my findings once the investigation is complete.

[snomiao]

let sno make a pr fixes this issue use the plan above

Investigation Report (by @snomiao)

🔍 Issue Analysis Complete

Root Cause: Both Chromatic and Playwright CI workflows fail at comment steps due to GitHub token permission restrictions, causing entire workflows to fail when commenting functionality is not essential for core testing.

📁 Files Analyzed

• .github/workflows/chromatic.yaml - Lines 33-51, 69-96 (comment steps)
• .github/workflows/test-ui.yaml - Lines 48-65, 135-148, 239-248, 325-332 (comment steps)

🎯 Technical Solution

The fix requires adding continue-on-error: true to all comment steps in both workflows to prevent GitHub API permission failures (403 errors) from blocking essential CI processes.

Specific Changes Needed:

1. chromatic.yaml: Add continue-on-error: true to:

   • "Comment PR - Build Started" (line 34)
   • "Comment PR - Build Complete" (line 70)

2. test-ui.yaml: Add continue-on-error: true to:

   • "Comment PR - Tests Started" (line 49)
   • "Comment PR - Browser Test Started" (line 136)
   • "Comment PR - Browser Test Complete" (line 240)
   • "Comment PR - Tests Complete" (line 326)

✅ Expected Outcome

• ✅ Chromatic visual testing continues even if comment fails
• ✅ Playwright browser testing continues even if comment fails
• ✅ Core CI functionality preserved
• ✅ Better developer experience (no failed CI due to comments)

🏗️ Implementation Ready

The solution is straightforward and low-risk - simply adding the continue-on-error: true parameter to existing comment steps will resolve the issue while maintaining all essential CI functionality.

[snomiao]

🔍 Investigation Started

I've started investigating your question/issue. Agent 11 is analyzing the codebase and will provide a detailed report soon.

• Agent ID: 11
• Status: Investigating
• Branch: issue-5149-ci-chromatic-comment-should-be-safe-to-fail

I'll update this comment with my findings once the investigation is complete.

[snomiao]

let sno make necessary code change and implementations and create draft pr

🔍 Investigation Steps by @snomiao

Step 1: Analyze Issue Context ✅

• Issue [ci] Chromatic comment should be safe to fail #5149: [ci] Chromatic comment should be safe to fail
• Root Cause: GitHub API permission restrictions (403 errors) causing CI workflows to fail at comment steps
• Impact: Essential CI processes (Chromatic builds, Playwright tests) blocked by non-critical comment failures

Step 2: Codebase Analysis ✅

• Analyzed - comment steps at lines 33-51, 69-96
• Analyzed - comment steps at lines 48-65, 135-148, 239-248, 325-332
• Identified 6 comment steps across both workflows that need

Step 3: Research Best Practices ✅

• Reviewed GitHub Actions documentation for parameter
• Confirmed this is the standard approach for non-critical workflow steps
• Verified this preserves core functionality while allowing optional steps to fail gracefully

Step 4: Implementation Plan ✅

Files to modify:

1. • Add to 2 comment steps
2. • Add to 4 comment steps

Step 5: Create Draft PR 🔄

• Implementing changes and creating draft PR with the solution

Status: Ready to implement and create PR

[snomiao]

🔍 Investigation Started

I've started investigating your question/issue. Agent 19 is analyzing the codebase and will provide a detailed report soon.

• Agent ID: 19
• Status: Investigating
• Branch: issue-5149-ci-chromatic-comment-should-be-safe-to-fail

I'll update this comment with my findings once the investigation is complete.