Skip to content
Merged
Prev Previous commit
Next Next commit
goal(FIX-1918): own reads the mode's own rail row before grading what…
… it leaves out

An empty, loading or erroring rail has no rows, so own passed without reading one.
It now waits for the walked session in its own rail (reopening Flow State's
assistant row, which a reload closes) and fails if it never shows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NczDiEMmwQjfux9CBTxNhE
  • Loading branch information
claude committed Oct 28, 2026
commit 8ee4677733f446f12cb2a6bf33bc201000712b52
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,4 @@ A control that leaves a must-leg green, or reddens a leg it doesn't name, fails
|------|--------|-------|---------|-------|
| 2026-10-28 | 56843f61cd | n/a | NOT GRADED (host) | Cloud container, root, no Codex or Cursor sign-in, no AI Gateway key. **cc:host**, **cx:host** and **cu:host** failed with the picker's reasons (root; Codex not logged in; no `CURSOR_API_KEY`), so no coding mode was walked. Flow State was walked, and its turns failed with no model (`fs:l`, `fs:c`, `fs:r`); `fs:own` passed. Needs a signed-in, non-root host (`fsd-qa`) |
| 2026-10-28 | 56843f61cd + this goal | scripted (`GOAL_SCRIPTED=1`) | dry run, not a verdict | All four modes walked on the scripted harnesses. The plumbing held: each tab's session and bound id read, the same id after turn 2 and after the reload in every harness mode, every tab back in its session after the reload, `cx:p` and `cx:kept` read "Codex reports no plan.", and **own** passed in all four. As expected for scripts, **c**, **r**, `cc:p` and `cu:p` failed: the scripts neither recall a token nor plan on a free prompt. `cc:l` failed too: the scripted Claude Code answers a free prompt faster than the walk polls (500 ms), so a real run is what grades it |
| 2026-10-28 | 56843f61cd + this goal, `own` reads its own row | scripted (`GOAL_SCRIPTED=1`) | dry run, not a verdict | **own** now first waits for the mode's own session in its rail, so a rail still loading, erroring or empty fails instead of passing. On the run before, `fs:own` had passed with no row at all: the reload had closed the assistant's rail row. With the row reopened, **own** passed in all four modes on their own rows. **l** failed in all four this time: the scripted turns finish inside the 500 ms poll, so the real run still grades it |
Original file line number Diff line number Diff line change
Expand Up @@ -373,6 +373,19 @@ await runGoal(async (failures, tell) => {
for (const walk of walks) {
const { page, mode } = walk;
const others = walks.filter((w) => w !== walk);
// A rail still loading, erroring or empty has no rows, so it would pass vacuously: its own session must show first.
if (!harness(walk)) {
// The reload closed the assistant's rail row, and its sessions are drawn only while it is open.
const row = shown(page, "rail").first().locator('button[data-kind="chat-agent"]');
if ((await row.getAttribute("aria-expanded").catch(() => null)) === "false") await row.click();
}
const ownRow = harness(walk)
? page.getByTestId("session-list").locator(`[data-session-id="${walk.sessionId}"]`)
: shown(page, "rail").first().locator(`[data-session-id="${walk.sessionId}"]`);
if (walk.sessionId === "" || !(await ownRow.first().waitFor({ state: "attached", timeout: 15_000 }).then(() => true, () => false))) {
fail(`${mode.tag}:own`, `the rail never listed this mode's own session ${JSON.stringify(walk.sessionId)}, so it can't show what it leaves out`);
continue;
}
const listed = harness(walk)
? await page.getByTestId("session-list").locator("[data-session-id]").evaluateAll((els) => els.map((el) => el.getAttribute("data-session-id") ?? ""))
: await page.locator("[data-session-id]").evaluateAll((els) => els.map((el) => el.getAttribute("data-session-id") ?? ""));
Expand Down
Loading