Skip to content

ci(release): packed-install checks DevTool serves its client assets (FIX-1334) - #2378

Merged
jhoffner merged 3 commits into
mainfrom
fix/fix-1334
Sep 29, 2026
Merged

jhoffner merged 3 commits into
mainfrom
fix/fix-1334

Conversation

@jhoffner

@jhoffner jhoffner commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

The problem

DevTool's page, scripts and styles (dist-client) are built by a separate step, not by the package's normal build. If a release leaves that step out, @flow-state-dev/devtool still publishes, and the first fsdev dev a user runs throws.

The release path already handles this: release:build builds the assets (#1844), and the publish refuses without them (#1846). The gap is in CI. No job installs a packed DevTool and asks it for its assets. pnpm pack skips the publish guard and packs a DevTool with no dist-client at exit 0, and every import check still passes. So nothing notices the break until release day.

What this does

It adds one check to the packed-install job (leg a) that FIX-1431 creates. It does not add a job. The check installs the packed DevTool into an empty project and does what fsdev dev does:

  1. asks the installed package for its asset directory, which must be inside node_modules;
  2. serves the directory beside the flow API;
  3. loads the page, then every script and stylesheet the page references.

It fails if any of those is missing or comes back as the wrong type.

Reproduced first (D2 / ER-17): the hole is already closed on main (5886ca846). The result and the commits that closed it are on the Linear issue. This PR is the guard that keeps it closed.

What's asked of you

Nothing here is yours to decide. This is a bug on the direct route, so this PR is the only gate.

Parts worth reviewing closely

  • scripts/packed-install/fixtures/serve-devtool.mjs: the content-type assertion. serve() answers every unknown path with index.html and a 200. Without the content-type check, an asset build that copied index.html but not assets/ would pass. Red state B below shows the check catching that case.
  • The containment check on getAssetPath(). It requires the asset directory to be inside the installed package. A leftover dist-client in the repo can't reach the consumer project anyway. The check is cheap and states the anti-game rule outright.
  • RELEASING.md. It gains one paragraph saying the publish hook doesn't cover pnpm pack and that this job does.

Stacks on #2376 (FIX-1431, leg a). Base is fix/fix-1431. When #2376 merges, this PR will be retargeted to main.

Linear FIX-1334 · epic FIX-1635 (specs/epics/FIX-1635/, ER-2) · merges after #2376

How to review this

How to review this. This is a bug fix on the direct route: there is no spec and no
spec PR, because the hard part was the diagnosis and it happened in code. This PR is the
only gate
, so unlike a spec-backed change, the approach is in scope. Nothing here was
signed off upstream.

Most valuable here: the diagnosis chain. Does the repro actually reproduce the reported
failure, is the named cause the real one, does the fix address that cause rather than the
symptom, and does the regression test fail without the fix? Then the second path (the legacy
shape, the null boundary, the concurrent case, the cancel path).

In scope to challenge: where the guard was placed, any behaviour change a user could
notice, and whether this should have been a spec'd feature instead of a fix.

Engineering calls
  • One more CHECKS entry, not a second job or a change to check-assets.mjs. This follows the epic (PLAN: "1431 creates it, 1334 adds to it, neither adds a second"). The prepublishOnly guard stays as it is. It covers the publish, and this check covers CI.
  • It serves the assets over HTTP instead of only checking that dist-client/index.html is in the tarball. ER-2 says DevTool serves its assets from an installed copy. A listing would pass a half-copied build.
  • A separate fixture, not an extension of serve.mjs. Each check names its own failure (devtool: FAIL at <step>), and serve.mjs stays FIX-1431's.
  • No unit test. The check has no judgment logic to pin, unlike classifyImport. It is the fixture, and red states A and B below exercise it.
  • No changeset. It changes only CI and a root script. No published package changes (BP-022).
Everything else: reproduction, control, verification

Reproduction on main 5886ca846 (does not reproduce):

Claim Evidence
The release build produces the assets rm -rf packages/devtool/dist-client && pnpm release:build: exit 0, dist-client/{index.html,assets} present
The packed tarball carries them pnpm pack in packages/devtool: 5 dist-client entries
The publish refuses without them dist-client moved aside, pnpm publish --dry-run: exit 1, refusing to publish without the DevTool app
A pack does not refuse same state, pnpm pack: exit 0, 0 dist-client entries

Closed by 9507b8a65 (#1844, release:build) and 2ab5d589b (#1846, prepublishOnly). The last commit with the hole is f27d9e31b, where every release script ran packages:build only.

Green (this branch, after pnpm release:build): node scripts/packed-install/run.mjs → ✓ packed install: 4 check(s) passed on 27 package(s), and devtool: ok (2 asset(s) from /tmp/fsd-packed-install-…/node_modules/@flow-state-dev/devtool/dist-client).

Red A: the pre-fix release build. This is f27d9e31b's release path, packages:build with no build:assets. It ran with TURBO_FORCE=true, because a local turbo cache replays a previously built dist-client as a build output. Result: exit 1. The three FIX-1431 checks pass and only this one fails:

• DevTool serves its client assets from the installed copy
  ✗ 1 failure(s):
    exit 1: devtool: FAIL at getAssetPath: @flow-state-dev/devtool: pre-built assets not found at /tmp/fsd-packed-install-h6VRg9/node_modules/@flow-state-dev/devtool/dist-client. …
✗ packed install: 1 of 4 check(s) failed

Red B: half-copied assets (index.html present, assets/ removed). Exit 1:

devtool: FAIL at asset: /assets/index-BkWP9cNV.js: content-type text/html; charset=utf-8, expected javascript

Other checks: node --check on both scripts. pnpm knip:ci exits 0. The existing packages/core/test/packed-install-check.test.ts passes (7/7). The --control step is unchanged: it runs only the import check.

Files

  • scripts/packed-install/run.mjs: one CHECKS entry, header list updated
  • scripts/packed-install/fixtures/serve-devtool.mjs: new, runs inside the consumer project
  • .github/workflows/ci.yml: step name only
  • docs/contributing/RELEASING.md: job description, plus the pack-vs-publish paragraph

🤖 Generated with Claude Code

https://claude.ai/code/session_01HNTQkGXCUryuVD2TEBLi6G


Generated by Claude Code


Note

Low Risk
Changes only CI scripts and release docs; no published package or runtime behavior changes.

Overview
Extends the packed-install release gate so CI catches a broken @flow-state-dev/devtool tarball even when pnpm pack skips prepublishOnly and import-only checks still pass.

A fourth check runs a new fixture (serve-devtool.mjs) in the empty consumer project: resolve getAssetPath() from the installed package, serve dist-client like fsdev dev, fetch index.html, and load every referenced script/stylesheet with content-type assertions (so SPA fallback cannot mask missing assets/). The CI step name and RELEASING.md are updated to document that this job—not publish hooks—covers the pack path.

Reviewed by Cursor Bugbot for commit 5f8e046. Configure here.

…FIX-1334)

The release path already builds DevTool's client assets (release:build,
#1844) and the publish refuses without them (prepublishOnly, #1846). What
nothing checked is the packed copy: `pnpm pack` skips prepublishOnly and
packs a devtool with no dist-client at exit 0, and every import still
passes, because only getAssetPath() needs the directory.

Appends one entry to the packed-install job's CHECKS list. It asks the
installed @flow-state-dev/devtool for its assets, requires them to sit
inside the installed package, serves them the way `fsdev dev` does, and
fetches index.html plus every script and stylesheet it references. It
checks each content type, because serve()'s SPA fallback answers a
missing asset with a 200 and index.html.

Red: on the pre-fix release build (packages:build only, as every release
script ran at f27d9e3), the other three checks pass and this one fails
at getAssetPath.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNTQkGXCUryuVD2TEBLi6G
@vercel

vercel Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
flow-state-dev-docs Ignored Ignored Sep 29, 2026 8:26pm UTC
flow-state-dev-kitchen-sink Ignored Ignored Sep 29, 2026 8:26pm UTC
flow-state-dev-ui Ignored Ignored Preview Sep 29, 2026 8:26pm UTC

Request Review

@linear-code

linear-code Bot commented Sep 29, 2026

Copy link
Copy Markdown

FIX-1334

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T20:13:38.611714Z 5f8e046 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copy link
Copy Markdown
Collaborator Author

Process guards is failing, and this PR did not cause it. The spec-folder guard fails on main itself, where it finds spec/_projects (merged in #2359). #2376 and #2377 fail it for the same reason. The owner has decided to retire that guard, and the change is in their hands. No fix to port yet.


Generated by Claude Code

Copy link
Copy Markdown
Collaborator Author

Second look + architecture pass

Verdict: the approach is right. Serving the installed assets over HTTP and asserting content-type is the smallest check that catches both the missing dist-client case and the half-copied case (the SPA fallback would otherwise mask it). One CHECKS entry rather than a new job matches the epic. I looked for a leaner shape (a tarball listing for dist-client/index.html) and it is weaker, so I wouldn't switch. Findings below are all small; none block.

Worth fixing in this PR

  1. The check can pass while checking almost nothing (serve-devtool.mjs, refs block). Two gaps:
    • It requires at least one <script src> but never requires a stylesheet. If the CSS link disappears from index.html (or its attributes come in a different order, since the regex needs rel="stylesheet" before href), the check passes with only the script verified. If the built page is expected to have a stylesheet, fail when there is none. Otherwise the "every stylesheet" claim in the PR body and in RELEASING.md is only true by luck of attribute order.
    • refs.some((r) => r.type.source === "javascript") compares against a regex's source string. Track kind: "script" | "style" on each ref instead.
  2. Absolute URLs. new URL(ref.url, base) will fetch an https://… reference from the network if index.html ever contains one (a CDN font or script). CI would then depend on the internet, and a passing fetch wouldn't prove anything about the package. Skip or fail on refs whose origin isn't base.
  3. Containment check. realpathSync(assetDir).startsWith(installed + sep) is fine, but installed comes from process.cwd(). That is correct because run.mjs spawns with cwd: project.dir; a one-line comment saying so would keep someone from "fixing" it with import.meta.url later.

Refactor opportunity (the PR is the second copy, so this is the moment)

  1. run.mjs now has two near-identical fixture runners (this entry and "a server starts and answers an action"): copyFileSync → spawnSync(process.execPath, …, {cwd, encoding, timeout: 60_000}) → success test on status === 0 && stdout.includes(marker) → find the <prefix>: FAIL line → exit N: verdict. A small runFixture(project, file, prefix) returning [] or [message] would collapse both to a few lines, and keeps "each check names its own failure" (the prefix). Only the success-log line differs. The PR body says serve.mjs stays FIX-1431's, but that is a reason to leave the fixture alone, not the runner. A follow-up to ci(release): pack, install and import every package in CI (FIX-1431) #2376 is fine if you'd rather not touch its leg here. A third fixture would make it clearly worth doing.
  2. Fixture boilerplate is duplicated between serve.mjs and serve-devtool.mjs (fail, the throwaway flow, createFlowState, serve(...)). Two ~20-line copies are tolerable as separate scripts run inside the consumer project, and I would not add a shared module there (it would need to be copied into the project too). The simpler cut is to let serve-devtool.mjs reuse a trivial flow. Not worth doing unless a third fixture appears.

Notes, no action

  • fail() calls process.exit(1) inside the try, so finally { handle.close() } does not run. Harmless (the process exits), just don't add cleanup that depends on it.
  • The options passed to serve() mirror packages/cli/src/commands/dev.ts. If dev grows options that affect asset serving (basePath, SPA fallback, a different staticDir resolution), this check won't follow automatically. A one-line pointer to dev.ts in the fixture header would help the next reader notice.
  • Docs: the RELEASING.md additions read fine and match the check's behaviour. Once Kitchen-sink: Add inline editable artifacts and saveArtifact action #1 is settled, make sure "every script and stylesheet" still describes what the fixture does.

No blocking findings. Items 1 and 2 are the ones I'd carry into a push; 4 is the refactor worth taking (here or as a follow-up).


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplify pass — verdict: ship it (with optional tidy-ups)

What was broken: pnpm pack skips prepublishOnly, so @flow-state-dev/devtool can tarball without dist-client while every import check still passes. getAssetPath() and fsdev dev are the first real consumer surfaces.

Why this approach is right: The fixture deliberately does what a consumer does — getAssetPath() from the installed package, serve() with staticDir, then HTTP — and it catches the SPA fallback footgun (missing asset → 200 + index.html). Tarball listing, re-running check-assets.mjs on the install tree, or prepack alone are all simpler but weaker; they do not prove serve() actually serves the bundled chunks with correct types. I would not replace this check with one of those.

Where complexity crept in (optional, not blocking):

  1. Fixture duplication — serve-devtool.mjs mirrors serve.mjs (fail, minimal flow, createFlowState, serve, try/finally). A shared fixtures/packed-serve-lib.mjs (copied alongside the entry script) or a single serve.mjs with a --devtool mode would shrink ~40 lines without changing behavior.
  2. run.mjs spawn harness — the new CHECK copies the same copyFileSync + spawnSync + verdict parsing as the server CHECK. A tiny runFixture({ file, okSubstring, failPrefix }) would keep timeout/encoding logic in one place.
  3. Flow boilerplate for static smoke — spinning up a noop flow is heavy, but it matches fsdev dev wiring (basePath, staticDir). I would not remove it unless @flow-state-dev/node gains a static-only helper (out of scope here).
  4. Fast-fail belt — if you want less CI noise when assets are totally missing, asserting dist-client/index.html (and maybe hashed chunks) inside the devtool tarball during packAll is cheap and complementary; it should stay in addition to this HTTP step, not instead of it.

Skipped as larger refactors: invoking fsdev dev / CLI discovery (breaks tarball-only isolation), HTML parsers in the consumer sandbox (new deps), merging into Playwright e2e (wrong layer).

No major concerns on performance (one extra Node cold start, sequential fetches on loopback with two assets today) or directory smell — one new fixture beside serve.mjs is consistent.

Approving; inline notes are nits you can take or leave.

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

Comment thread scripts/packed-install/run.mjs Resolved
Comment thread docs/contributing/RELEASING.md Resolved
Brings in #2376's review round (importPass, wildcard expansion, the vitest
probe fixture, resolvedGraph removed). No conflicts: the DevTool check is
appended after the server check, which the base did not touch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNTQkGXCUryuVD2TEBLi6G

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5f8e04635c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Snob. A smaller shape meets the same contract. Fold it from #2379 and close that POC unmerged after this PR lands. Do not merge the POC.

Two cuts, both in the new check:

  1. serve-devtool.mjs builds a noop handler, two Zod schemas, and never calls the action. defineFlow({ kind, actions: {} }) is enough to hand serve() a flow. Drop handler and zod.
  2. The new CHECKS entry copies the server check's copyFileSync / spawnSync / ": ok" / ": FAIL" block. One runFixture(project, file, stem) runs both. Each fixture still names its own failure, and serve.mjs stays as it is.

The check-entry comment restates the fixture. One line is enough: pnpm pack skips prepublishOnly.

Left alone on purpose: the separate fixture (already decided on this PR), the content-type walk, and the containment check. Those are the contract.

The POC is 35 insertions and 63 deletions on top of this head. Green still prints devtool: ok (2 asset(s) …). Missing dist-client still fails at getAssetPath. index.html without assets/ still fails on content-type text/html for the script.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Code Snob

Review round 1 on #2378.

- Codex P2: the check fetched only what index.html references directly.
  Assets the bundle itself references, like the two SVG marks, could be
  missing and still pass, because the SPA fallback answers them with
  index.html. The fixture now also fetches every `/assets/` file named
  in a fetched script or stylesheet, and checks each one's type. It
  fails on a build with the SVGs removed; the previous fixture passed it.
- Cursor and Code Snob: one runFixture(project, file, stem) helper
  shared by the serve and DevTool checks, and a one-line CHECKS comment.
  The flow is defineFlow({ kind, actions: {} }) with no handler or zod.
  realpathSync is called once. Asset kinds are plain "js" | "css" |
  "svg" | "other" instead of RegExp pseudo-enums.

Folds POC #2379.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNTQkGXCUryuVD2TEBLi6G
@jhoffner
jhoffner changed the base branch from fix/fix-1431 to main September 29, 2026 22:29
@jhoffner
jhoffner merged commit 12d8cff into main Sep 29, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants