Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
FIX-2335: Teammates lab drops projects for one working folder from co…
…nfig

The lab's host now runs on the existing constant run source, a git checkout
the operator names in TEAMMATES_WORKDIR, listed in localRepositories. Unset,
missing or not a checkout, the lab fails at start; it never falls back to the
current directory or its own checkout. Project tools, TEAMMATES_PROJECTS and
the start-up project writes are gone, and so are the header project select,
the New project form, the needs-project card and the repository approval card.
Task cards say running and done. No WORKER.md mentions a project. ER-8 and
FIX-2154 D3 get dated amendments scoped to this lab; packages/ is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NczDiCsRqCenZsFQRpJBHi
  • Loading branch information
claude committed Oct 29, 2026
commit e117f54c714f9d657ebcc9f870526deb00a8cfd9
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@
"What is a good habit for better sleep, in one line?"
],
"pastWindow": 60,
"project": { "id": "relay", "title": "Relay" },
"typos": [
{ "readme": "# Relay — a tiny relay servce", "wrong": "servce", "right": "service" },
{ "readme": "# Relay — passes mesages along", "wrong": "mesages", "right": "messages" },
Expand Down
Original file line number Diff line number Diff line change
@@ -1,26 +1,25 @@
# teammates-lab › it a worker talks to another while you talk

**Issue:** FIX-2154 (leg b of the FIX-2152 epic's goal); step 5 and the held-out date amended by FIX-2309, step 5's coding task by FIX-2332; step 4 with FIX-1613
**Issue:** FIX-2154 (leg b of the FIX-2152 epic's goal); step 5 and the held-out date amended by FIX-2309, step 5's coding task by FIX-2332, and its project by FIX-2335 (the lab has none); step 4 with FIX-1613
**Outcome:** In the Teammates lab, a list-of-workers app on the Grok Bot pattern, you talk to lead, lead messages researcher, and researcher's reply reaches lead while you are still talking to it; lead uses it. Long after both have left lead's history window, lead still knows them, and the screen shows where its word-for-word reading begins. A coding task lead starts runs in the background through Claude Code, in the user's one project, which lead puts its conversation in without asking.
**Input:** held out. The launch date is drawn at random each run and written into the project's README, which researcher reads and lead can't, so lead can only state it by hearing it from researcher. Step 4's codename (a colour, an animal and a number) is drawn per run, and its 60 lines are shuffled. The README's typo is one of several (`fixtures/input.json`). The lines you send are in the fixture.
**Input:** held out. The launch date is drawn at random each run and written into the working folder's README, which researcher reads and lead can't, so lead can only state it by hearing it from researcher. Step 4's codename (a colour, an animal and a number) is drawn per run, and its 60 lines are shuffled. The README's typo is one of several (`fixtures/input.json`). The lines you send are in the fixture.
**Signal:**
- **1:** lead answers your first line in its conversation.
- **2:** a card *To researcher* in lead's conversation, a line *From lead* in researcher's, the exchange under *with researcher* in lead's Talk panel, and a delivered row lead → researcher in the message log.
- **3:** your second line goes in before researcher's reply has landed. The reply shows in lead's conversation under researcher's name, after lead's answer to you, carrying the date; the log holds it as a delivered reply; a line of lead's after it states the date.
- **4:** you tell lead a codename, then send 60 more lines, past the 50-turn history window, each answered before the next. Then you ask for the codename and the launch date. lead's answer holds all three words of the codename and the date, and lead sent researcher nothing new for it (no new card, no new log row). The conversation shows a divider labelled *Earlier turns summarized*, and both the codename line and researcher's reply are drawn above it.
- **5:** you ask for the fix. With one project, lead puts its conversation in `relay` and files the task there: no card asks for a project, and the header shows `relay`. The coding task's run ends *done in relay*, its steps show Claude Code, and the result lands in lead's conversation as the coding task's. On disk: the checkout's HEAD and working tree are untouched, and exactly one worktree of it, under the lab's runs folder, carries a commit that changes README.md alone, fixing its first line and nothing else.
- **5:** you ask for the fix. lead starts a coding task with no question: no card asks where, and no project shows anywhere on the page, nor in any worker's line. The coding task's run ends *done*, its steps show Claude Code, and the result lands in lead's conversation as the coding task's. On disk: the working folder's HEAD and working tree are untouched, and exactly one worktree of it, under the lab's runs folder, carries a commit that changes README.md alone, fixing its first line and nothing else.
**Anti-game:** Graded on what the browser shows, the message log read through the public client, and git. Never on the lab's adapter or the server's own state. Lead can't state the launch date without the reply, because it is drawn per run. The disk check reads git, so a run that only says it fixed the typo fails. The step-3 order check reads entry order as drawn, so a reply drawn before lead's answer fails. Step 4 counts lead's messages to researcher around the question, so asking researcher again for the date fails, and it reads which side of the divider each fact sits on, so a fact still read word for word doesn't count. Each graded list is floored with `tell.over`; an empty log or no worktree counts only where a control produces it, and says so.
**Model:** real — openai/gpt-5.4-mini (lead, researcher, writer); coder's run on Claude Code
**Run:** `pnpm tsx goals/teammates-lab/a-worker-talks-to-another-while-you-talk/run.mts` (needs `AI_GATEWAY_API_KEY`, and Claude Code signed in or `ANTHROPIC_API_KEY`)
**Controls:** each on the same command, failing the steps named and no other:
- `GOAL_CONTROL=no-model`: the workers answer on a model that doesn't exist. Steps 1, 2, 3 and 5 FAIL.
- `GOAL_CONTROL=no-lead-send`: lead's `sendMessage` is dropped from its tools (a scratch patch to `message-tool.ts`). Steps 2 and 3 FAIL. Step 5 may fail too: with no way to message, lead files its research ask as a task for coder, and after the pick it filed nothing again (2 of 2 runs).
- `GOAL_CONTROL=no-reply-tool`: researcher's `sendMessage` is dropped. Step 3 FAILS.
- `GOAL_CONTROL=project-not-kept`: `setWorkProject` answers as kept but stores nothing (a scratch patch to `work-project.ts`), so lead's task has no project and the README never changes. Step 5 FAILS. Step 3 may fail too: researcher can't keep the project it reads the date from.
- `GOAL_CONTROL=no-summary`: the history summary's capability and fold are dropped from the lab's host (a scratch patch to `host.mts`); memory stays. Step 4 FAILS: the divider can't say *summarized*.
- `GOAL_CONTROL=no-memory`: the summary and memory are both dropped from the host. Step 4 FAILS: nothing holds what left the window.

Step 4 is skipped under `no-model`, `no-lead-send`, `no-reply-tool` and `project-not-kept`: its 60 lines test nothing about the step each control reddens.
Step 4 is skipped under `no-model`, `no-lead-send` and `no-reply-tool`: its 60 lines test nothing about the step each control reddens.

Steps 2 and 3 have no isolating red state for lead's side: without lead's message there is nothing to reply to, so `no-lead-send` reddens both. Under `no-model` the screen doesn't show your own lines either until a reload: a turn that fails shows nothing live.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
* Goal check: in the Teammates lab, you talk to lead, lead messages
* researcher, researcher's reply reaches lead while you are still talking to
* it, and a coding task lead starts runs through Claude Code, in the
* background, in lead's project. See goal.md for the contract.
* background, in the lab's working folder. See goal.md for the contract.
*
* Real path, real model, out of CI. The lab's pages are built here and served
* by `fsdev dev --app` over a fresh SQLite store, with one project, `relay`,
* mapped to a scratch git checkout made here. One headless browser drives the
* by `fsdev dev --app` over a fresh SQLite store, with a scratch git checkout
* made here as its working folder (`TEAMMATES_WORKDIR`). One headless browser drives the
* screen as a person would. Graded, per step:
*
* 1 lead answers your first line in its conversation.
Expand All @@ -21,18 +21,18 @@
* lead answers the codename and the launch date without asking anyone,
* and the divider, labelled summarized, sits after both the codename
* line and researcher's reply.
* 5 you ask for a fix; with one project, lead puts its conversation in
* `relay` and files the task there with no question, the run goes through
* 5 you ask for a fix; lead starts a coding task with no question and no
* project anywhere on the screen, the run goes through
* Claude Code, a worktree of the checkout gets one commit that changes the
* README's first line and nothing else, the checkout itself is untouched,
* and the result lands in lead's conversation as the coding task's.
*
* Inputs are held out: the launch date is drawn at random and written into the
* project's README, where researcher reads it, the codename is drawn at random, and the
* working folder's README, where researcher reads it, the codename is drawn at random, and the
* README's typo is one of several.
*
* Run: pnpm tsx goals/teammates-lab/a-worker-talks-to-another-while-you-talk/run.mts
* Controls: GOAL_CONTROL=no-model | no-lead-send | no-reply-tool | project-not-kept | no-summary | no-memory (see goal.md)
* Controls: GOAL_CONTROL=no-model | no-lead-send | no-reply-tool | no-summary | no-memory (see goal.md)
*/
import { spawn, execFileSync, type ChildProcess } from "node:child_process";
import { writeFileSync, mkdirSync } from "node:fs";
Expand Down Expand Up @@ -72,7 +72,6 @@ interface Fixture {
fillers: string[];
/** How many lines step 4 sends between the codename and the question: past the 50-turn window. */
pastWindow: number;
project: { id: string; title: string };
typos: { readme: string; wrong: string; right: string }[];
rest: string;
answerWithinMs: number;
Expand Down Expand Up @@ -107,25 +106,11 @@ const CONTROLS: Record<string, { patches?: ModulePatch[]; model?: string; steps:
// No model answers: lead's conversation holds only your own lines.
"no-model": { model: "openai/no-such-model-goal-control", steps: ["1", "2", "3", "5"], skip: ["4"] },
// lead can't message: no card, no line in researcher's conversation, no row, and so no reply.
// Step 5 may go red too: with no way to message, lead files its research ask as a task instead,
// and after the pick it has filed none again (seen in 2 of 2 runs). That is the model, not a leak.
// Step 5 may go red too: with no way to message, lead files its research ask as a task instead
// (seen in 2 of 2 runs before the lab dropped projects). That is the model, not a leak.
"no-lead-send": { patches: [dropSendFor(fixture.lead)], steps: ["2", "3"], mayAlso: ["5"], skip: ["4"] },
// researcher hears lead but can't answer it.
"no-reply-tool": { patches: [dropSendFor(fixture.researcher)], steps: ["3"], skip: ["4"] },
// A conversation's project answers as kept but is never stored, so lead's task has no project.
// Step 3 may go red too: researcher puts its own conversation in the project to read the date,
// and that is not stored either.
"project-not-kept": {
patches: [
{
module: "packages/workforce/src/projects/work-project.ts",
from: String.raw`await ctx\.session\.patchState\(\{ \[WORK_PROJECT_STATE_KEY\]: projectRef\(input\.project\) \}\);`,
to: "/* goal control project-not-kept: not stored */",
},
],
steps: ["5"],
skip: ["4"],
},
// Step 3 may go red under both of these: researcher sometimes answers lead in its own conversation
// instead of with `sendMessage`, so no reply lands (2 of 3 runs on 2026-10-28). Neither patch
// touches messaging; that is the model, not a leak.
Expand Down Expand Up @@ -391,7 +376,7 @@ await runGoal(async (failures, tell_) => {
TEAMMATES_MODEL: control?.model ?? MODEL,
TEAMMATES_STORE: join(dir, "teammates.sqlite"),
TEAMMATES_USER: "you",
TEAMMATES_PROJECTS: JSON.stringify([{ ...fixture.project, checkout }]),
TEAMMATES_WORKDIR: checkout,
...(patches.length > 0 ? modulePatchEnv(patches, REPO_ROOT, mark) : {}),
});
console.error(`goal: launch date ${date.text}; typo "${typo.wrong}"; scratch ${dir}`);
Expand Down Expand Up @@ -506,7 +491,7 @@ await runGoal(async (failures, tell_) => {
if (control?.skip?.includes("4")) evidence.push(`4: not run under GOAL_CONTROL=${CONTROL}`);
else await stepFour(page, log, date, fail, evidence, tell_);

// ---- 5 · lead starts a coding task for a fix, in the one project ---------
// ---- 5 · lead starts a coding task for a fix, in the working folder ------
const before = diskOf(checkout);
await tell(page, lead, fixture.fix);
const tasks = conversation(page).locator("[data-entry=task]");
Expand All @@ -516,18 +501,17 @@ await runGoal(async (failures, tell_) => {
const fixAt = (es: Drawn[]) => userAt(es, fixture.fix);
const ended = await readUntil(
page,
(es) => cardsNow(es).some((e) => e.status !== undefined && /^(done in|failed|refused|needs)/.test(e.status)),
(es) => cardsNow(es).some((e) => e.status !== undefined && /^(done|failed)$/.test(e.status)),
fixture.runWithinMs,
);
const filed = tell_.over("5 · task cards lead filed in this step", cardsNow(ended.entries), { emptyIs: "lead filed nothing, which a control produces" });
const run = filed.find((e) => e.status !== undefined && /^(done in|failed|refused|needs)/.test(e.status));
const run = filed.find((e) => e.status !== undefined && /^(done|failed)$/.test(e.status));
if (filed.length === 0) fail("5", `lead started no coding task within ${fixture.runWithinMs / 1000}s`);
else if (run?.status !== `done in ${fixture.project.id}`) fail("5", `lead's task ended "${run?.status ?? filed[0]!.status}", not "done in ${fixture.project.id}"`);
// One project: lead files into it without asking which (FIX-2309's check 2), and never sends a card that waits for a pick.
if (filed.some((e) => e.status === "needs a project")) fail("5", "a task card asked for a project though there is only one");
const chip = await page.getByLabel(`Project for ${lead}`).inputValue();
if (chip !== fixture.project.id) fail("5", `lead's conversation shows project "${chip || "none"}", not "${fixture.project.id}"`);
if (run?.status === `done in ${fixture.project.id}`) {
else if (run?.status !== "done") fail("5", `lead's task ended "${run?.status ?? filed[0]!.status}", not "done"`);
// The lab has no projects (FIX-2335): the page never shows one, and no worker says the word.
const said = (await page.locator("body").innerText()).match(/[^\n]*\bprojects?\b[^\n]*/i);
Comment thread
jhoffner marked this conversation as resolved.
Resolved
if (said !== null) fail("5", `the page mentions a project: "${said[0].slice(0, 200)}"`);
if (run?.status === "done") {
// The run, opened on its card: it went through Claude Code.
const runCard = tasks.nth(earlier + filed.indexOf(run));
await runCard.getByRole("button", { name: /^Coding task: / }).click();
Expand Down Expand Up @@ -567,7 +551,7 @@ await runGoal(async (failures, tell_) => {
// `git` here trims the output, so the README's last newline is gone.
if (readme.slice(1).join("\n") !== rest.trimEnd()) fail("5", "the run changed more of the README than its first line");
if (!worktree!.path.startsWith(runsRoot)) fail("5", `the run's worktree ${worktree!.path} is not under the lab's runs folder`);
evidence.push(`5: filed in ${fixture.project.id} with no question; Claude Code fixed "${typo.wrong}" in ${worktree!.path}, README.md only, checkout untouched`);
evidence.push(`5: started with no question and no project; Claude Code fixed "${typo.wrong}" in ${worktree!.path}, README.md only, checkout untouched`);
}
if (patches.length > 0) assertPatched(patches, REPO_ROOT, mark);
} finally {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@
*
* Run: pnpm tsx goals/teammates-lab/consults-a-conversation-without-waking-it/run.mts
*/
import { spawn, type ChildProcess } from "node:child_process";
import { execFileSync, spawn, type ChildProcess } from "node:child_process";
import { randomInt } from "node:crypto";
import { cpSync, readFileSync, writeFileSync } from "node:fs";
import { cpSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import { DatabaseSync } from "node:sqlite";
import { createClient, readEveryCollectionPage } from "@flow-state-dev/client";
Expand Down Expand Up @@ -94,8 +94,11 @@ type Item = {
content?: { text?: string }[];
};

/** The teammates lab, served from a copy of its workers' tree. */
/** The teammates lab, served from a copy of its workers' tree, over an empty scratch checkout as its working folder. */
async function startTeammates(dir: string, tree: string, env: Record<string, string>): Promise<{ child: ChildProcess; origin: string }> {
const workdir = join(dir, "workdir");
mkdirSync(workdir, { recursive: true });
execFileSync("git", ["init", "-q", workdir]);
const port = 46_000 + randomInt(0, 9_000);
const origin = `http://127.0.0.1:${port}`;
await refuseIfAnswering(origin);
Expand All @@ -109,6 +112,7 @@ async function startTeammates(dir: string, tree: string, env: Record<string, str
TEAMMATES_STORE: join(dir, "teammates.sqlite"),
TEAMMATES_TREE: tree,
TEAMMATES_USER: USER,
TEAMMATES_WORKDIR: workdir,
...env,
}),
});
Expand Down
Loading
Loading