Repository navigation
FIX-2087: closure QA for one task primitive: the walk, the sweep in CI, and 13 findings #3692
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
3d7effe
test(FIX-2087): the closure's goal and the task-surface sweep in CI
claude 63da04f
test(FIX-2087): the closure's first verdicts
claude f6d06f1
test(FIX-2087): the sweep reads tools and endings by symbol; controls…
claude b6f75bf
test(FIX-2087): a forwarder exempts only its forwarding; every block …
claude fdb18c5
Merge remote-tracking branch 'origin/main' into fix/FIX-2087
claude b76f76b
test(FIX-2087): j1 counts only notice requests that ran a turn; j1:he…
claude e7c24c7
Merge remote-tracking branch 'origin/main' into fix/FIX-2087
claude 7e07954
Merge remote-tracking branch 'origin/main' into fix/FIX-2087
claude deaccbd
test(FIX-2087): row 3 grades the closure's own findings
claude 1ba899d
test(FIX-2087): classify the closure goal in FIX-1792's census
claude 89217e4
test(FIX-2087): the closure's verdict on main with every closure fix …
claude 024a259
Merge remote-tracking branch 'origin/main' into fix/FIX-2087
claude 5e5ee0e
test(FIX-2087): row 3 grades FIX-2123, FIX-2135 and FIX-2136
claude 455b3fb
test(FIX-2087): a hearing is a notice admitted, not a request started
claude 1defdcc
Merge remote-tracking branch 'origin/main' into fix/FIX-2087
claude e51ac8b
test(FIX-2087): row 3 runs FIX-2138's regression test
claude cde0a95
test(FIX-2087): round 3's verdicts
claude 46a6629
Merge origin/main into fix/FIX-2087
claude 4ec08e6
test(goals): grade FIX-2147, FIX-2149 and FIX-2150 in the closure's r…
claude 8a5ed55
refactor(goals): fold the shorter one-task notes from #3790 (083cef8a…
claude 77dd49e
fix(goals): the closure fails closed, and grades a word-for-word foll…
claude 683c151
fix(goals): every control's tags must go red, and the door rule reads…
claude 9a35628
fix(goals): read a control's red from its tagged failures, and bring …
claude b6516a8
docs(goals): the closure's goal states how a control is read and what…
claude eb33735
docs(goals): round 4's verdicts in the closure's log (FIX-2087)
claude File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next
Next commit
test(FIX-2087): the closure's goal and the task-surface sweep in CI
goals/workforce-coordination/runs-on-one-task/ runs the closure's plan on one main commit: the concept's walk (leg a, with journeys j1 and j2), the server killed after an ending (b) and a task failing out (c), on a real model, on a Lab of WORKER.md files served by Shift Manager's command; the task-surface sweep; every child's check with its controls; and the 31 in-scope gaps rows' map. Each assertion is tagged structural or judgment, and a structural miss on any attempt is never retried past (D1). The sweep is one rule engine (rules.mjs, engine.mjs) the goal and CI share (D2): the tools rule reads every tool the task-tools capability declares, the removed rule reads the epic's migration table as it stands, and the endings rule reads callers of a task collection's complete and fail, and claim-passing cancels, by type through the TypeScript program. --control plants a sixth tool, a removed name, a write through a renamed variable and a claim-passing cancel. It joins CI's Typecheck job, which installs the TypeScript it reads with. FIX-2078's and FIX-2080's controls move to a controls.mts beside their goals, so the closure reuses them rather than copying them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
- Loading branch information
commit 3d7effe91c9d70fd95ee5ce5f7682c2517ed07a4
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
25 changes: 25 additions & 0 deletions
25
goals/task-board/every-ending-reaches-its-filer/controls.mts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| /** | ||
| * This goal's named control and its kill point, as scratch patches (`goals/lib/module-patch.mts`). | ||
| * Exported so a goal that composes this one (FIX-2087's closure walk) reuses them rather than | ||
|
jhoffner marked this conversation as resolved.
Resolved
|
||
| * copying them. | ||
| */ | ||
| import type { ModulePatch } from "../../lib/index.mts"; | ||
|
|
||
| /** Each named control, by its `GOAL_CONTROL` name. */ | ||
| export const PATCHES: Record<string, ModulePatch[]> = { | ||
| // The sweeper fires no wake: only a fast path or a touch pays a debt. | ||
| "no-sweeper-payment": [ | ||
| { | ||
| module: "packages/engine/src/durability/durability-sweeper.ts", | ||
| from: String.raw` if \(fireWake === undefined \|\| wakes === undefined\) return;`, | ||
| to: " return;" | ||
| } | ||
| ] | ||
| }; | ||
| /** Leg b's second process dies the moment an ending's delivery would send a notice it owes. */ | ||
| export const KILL_AFTER_ENDING: ModulePatch = { | ||
| module: "packages/orchestration/src/tasks/notice/delivery.ts", | ||
| // The first line of the delivery's send half, after the ending's write. | ||
| from: String.raw` const notices = owedNotices\(row, board\.collectionId, rows, filerOf\);`, | ||
| to: ' if (process.env.GOAL_KILL === "after-ending" && Object.keys(row.metadata ?? {}).some((k) => k.startsWith("noticeOwed:"))) process.exit(0); const notices = owedNotices(row, board.collectionId, rows, filerOf);' | ||
| }; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| /** | ||
| * This goal's named control, as scratch patches (`goals/lib/module-patch.mts`). Exported so a | ||
| * goal that composes this one (FIX-2087's closure walk) reuses it rather than copying it. | ||
| */ | ||
| import type { ModulePatch } from "../../lib/index.mts"; | ||
|
|
||
| /** Each named control, by its `GOAL_CONTROL` name. */ | ||
| export const PATCHES: Record<string, ModulePatch[]> = { | ||
| // Both cascades off: no ending records the debt, so nothing pays one, and a | ||
| // touch doesn't sweep for dependents an ending missed. | ||
| "no-cascade": [ | ||
| { | ||
| module: "packages/orchestration/src/tasks/collection/internal.ts", | ||
| from: String.raw` if \(recording\?\.owesCascade\?\.\(row as Task, ending\) !== true\) return row;`, | ||
| to: " return row;" | ||
| }, | ||
| { | ||
| module: "packages/orchestration/src/tasks/collection/cascade.ts", | ||
| from: String.raw` await cancelDeadDependents\(writes, now\);`, | ||
| to: "" | ||
| } | ||
| ] | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,144 @@ | ||
| /** | ||
| * The concept's cast, written as `WORKER.md` files at run time, so the topic, the held-out | ||
| * word and the researcher's fact are picked per run and nothing in the repository holds them. | ||
| * | ||
| * - `desk.launch`: the built-in coordinator, routing everyone, its one delegate `desk.writer`. | ||
| * - `desk.writer`: the built-in `agent` worker, delegate `desk.researcher`. It never knows the | ||
| * fact: it asks researcher, then fact-checks the fact in a step of its own run in a session of | ||
| * its own (`session: "new"`), then writes the draft. | ||
| * - `desk.researcher`: the built-in `agent` worker, no delegates. Only its file holds the fact. | ||
| * - `desk.chief` and `desk.editor`, for the team that hands out work (journey j1): a coordinator | ||
| * routing everyone to `desk.editor`, which files for researcher and waits on it with | ||
| * `getTask` in the same request. | ||
| * | ||
| * Leg c's tree (`variant: "fails"`) gives researcher a model that refuses every call, and has | ||
| * writer file a second research task that depends on the first, so the first's failure has a | ||
| * dependent to cancel. | ||
| */ | ||
| import { randomBytes, randomInt } from "node:crypto"; | ||
| import { mkdirSync, writeFileSync } from "node:fs"; | ||
| import { join } from "node:path"; | ||
|
|
||
| /** What a run picked: Alice's topic and words, and the fact only researcher's file holds. */ | ||
| export interface Cast { | ||
| topic: string; | ||
| /** The held-out word Alice asks the draft to carry. */ | ||
| word: string; | ||
| /** A second held-out word, sent only as a steer (journey j2). */ | ||
| steerWord: string; | ||
| fact: string; | ||
| /** The digits in the fact, which a draft that used it carries. */ | ||
| factMark: string; | ||
| } | ||
|
|
||
| const TOPICS = ["a solar lantern", "a folding kayak", "a pocket espresso maker", "a trail camera", "a smart plant pot", "a cordless glue gun"]; | ||
| const WORDS = ["heron", "copper", "lantern", "marble", "saffron", "willow", "ember", "glacier", "orchid", "tinsel"]; | ||
| const pickWord = () => `${WORDS[randomInt(WORDS.length)]}${randomBytes(2).toString("hex")}`; | ||
|
|
||
| /** Pick a topic, the held-out words and a fact for one run. */ | ||
| export function pickCast(): Cast { | ||
| const hours = 100 + randomInt(800); | ||
| return { | ||
| topic: TOPICS[randomInt(TOPICS.length)]!, | ||
| word: pickWord(), | ||
| steerWord: pickWord(), | ||
| fact: `its battery lasts ${hours} hours on one charge`, | ||
| factMark: String(hours), | ||
| }; | ||
| } | ||
|
|
||
| /** The ids the cast goes by. */ | ||
| export const CAST = { | ||
| launch: "desk.launch", | ||
| writer: "desk.writer", | ||
| researcher: "desk.researcher", | ||
| chief: "desk.chief", | ||
| editor: "desk.editor", | ||
| } as const; | ||
|
|
||
| const FIRST_STEP = { | ||
| walk: `1. If you have not heard back from desk.researcher yet, file exactly one task for it with createTask (assignee "desk.researcher") asking for the product's key fact, and end your turn with one short line saying you are waiting on research. Don't write anything else yet, and don't wait on the task.`, | ||
| fails: `1. If you have not heard back from desk.researcher yet, file two tasks for it, both assignee "desk.researcher", one call at a time. First call createTask alone, asking for the product's key fact, and read the taskId it returns. Only then call createTask a second time, asking for a customer quote about that fact, with deps: [that taskId]. Then end your turn with one short line saying you are waiting on research. Don't write anything else yet, and don't wait on either task.`, | ||
| }; | ||
|
|
||
| /** | ||
| * Write the tree under `dir` (`teams/desk/workers/<id>/WORKER.md`). | ||
| * @param variant `walk` (legs a and b, the journeys) or `fails` (leg c) | ||
| */ | ||
| export function writeCast(dir: string, cast: Cast, variant: "walk" | "fails" = "walk"): string { | ||
| const worker = (id: string, text: string) => { | ||
| const at = join(dir, "teams", "desk", "workers", id); | ||
| mkdirSync(at, { recursive: true }); | ||
| writeFileSync(join(at, "WORKER.md"), text, "utf8"); | ||
| }; | ||
| worker( | ||
| "launch", | ||
| `--- | ||
| description: Takes launch writing requests and hands them to the writer. | ||
| flow: coordinator | ||
| routing: everyone | ||
| model: goal/real | ||
| delegates: [desk.writer] | ||
| --- | ||
|
|
||
| Hand each request to the writer. | ||
| `, | ||
| ); | ||
| worker( | ||
| "writer", | ||
| `--- | ||
| description: Writes launch posts. Asks desk.researcher for product facts. | ||
| model: goal/real | ||
| delegates: [desk.researcher] | ||
| --- | ||
|
|
||
| You are desk.writer. You write short launch posts. You never know a product fact yourself: desk.researcher does. | ||
|
|
||
| If what you are given starts with "Fact-check", it is a step you filed for yourself: file nothing, call no tool, and answer only "Checked: " followed by the fact it names. Nothing else. | ||
|
|
||
| When you are asked to draft a launch post: | ||
| ${FIRST_STEP[variant]} | ||
| 2. When desk.researcher's task comes back with the fact, file exactly one step for yourself with createTask: assignee "desk.writer", session "new", goal "Fact-check: " followed by the fact as researcher stated it. Then end your turn with one short line. Don't write the draft yet. | ||
| 3. When you are told your steps have ended, answer with the finished post: three sentences at most, using the fact exactly as researcher stated it and every word the person asked for, including any word they added while you worked. | ||
|
|
||
| If research fails, answer with one line saying the post can't be written without the fact. | ||
|
|
||
| When you are asked for a new version of a post you already wrote, write it from the earlier version, keeping the fact and the person's words. | ||
| `, | ||
| ); | ||
| worker( | ||
| "researcher", | ||
| `--- | ||
| description: Knows the product facts. | ||
| model: ${variant === "fails" ? "goal/broken" : "goal/real"} | ||
| --- | ||
|
|
||
| You are desk.researcher. The product fact you know: ${cast.fact}. When you are asked for a fact, answer with that fact in one sentence, exactly as written here. When you are asked for a customer quote, answer with one short made-up quote that mentions the fact. | ||
| `, | ||
| ); | ||
| worker( | ||
| "chief", | ||
| `--- | ||
| description: Takes fact questions and hands them to the editor. | ||
| flow: coordinator | ||
| routing: everyone | ||
| model: goal/real | ||
| delegates: [desk.editor] | ||
| --- | ||
|
|
||
| Hand each question to the editor. | ||
| `, | ||
| ); | ||
| worker( | ||
| "editor", | ||
| `--- | ||
| description: Answers product questions by asking desk.researcher. | ||
| model: goal/real | ||
| delegates: [desk.researcher] | ||
| --- | ||
|
|
||
| You are desk.editor. You never know a product fact yourself. When you are asked a question about the product, file exactly one task for desk.researcher with createTask asking it, then wait for it in this same turn with getTask, passing the taskId and wait: true. When it comes back, answer in one sentence with the fact exactly as researcher stated it. | ||
| `, | ||
| ); | ||
| return dir; | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.