From 8563d524024760f0b04b580a8e7b7d425141f964 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 22 Sep 2026 01:26:05 +0000 Subject: [PATCH 1/3] =?UTF-8?q?spec(FIX-1496):=20ER-DevForce=20proof=20?= =?UTF-8?q?=E2=80=94=20thinnest=20DevForce=20path=20ships=20a=20real=20art?= =?UTF-8?q?ifact?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The retained spec set for W5's ER-DevForce exit proof, framed on the research finding that the proof is nearly built and has never been run: the model-backed check `it-commits-from-the-seats-own-file` is complete, type-checked and its verdict log's one row reads NOT RUN. Four gaps close, and nothing new is built. The declared feature channel is opened and driven rather than merely walked; the run moves to durable on-disk stores; the artifact becomes a pull request at an address that resolves after the run, instead of a commit in a temp directory; and the brief gains an executable acceptance condition so the product is graded, not just the prompt. Includes poc/gap-check/, a Node script that re-derives the twelve facts the spec rests on straight from the repository — with a totality assertion over every TypeScript file in the lab and four planted defects each watched going red. All twelve green, including that FIX-1440 does not fence this work. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016qaz1VpRK4wCQzxpjzZ4Hz --- specs/issues/FIX-1496/BUSINESS-RULES.md | 65 +++++ specs/issues/FIX-1496/DECISIONS.md | 119 ++++++++ specs/issues/FIX-1496/DOCS.md | 76 +++++ specs/issues/FIX-1496/PLAN.md | 141 ++++++++++ specs/issues/FIX-1496/SPEC.md | 103 +++++++ .../issues/FIX-1496/figures/the-four-gaps.svg | 101 +++++++ specs/issues/FIX-1496/poc/gap-check/README.md | 56 ++++ specs/issues/FIX-1496/poc/gap-check/check.mjs | 264 ++++++++++++++++++ 8 files changed, 925 insertions(+) create mode 100644 specs/issues/FIX-1496/BUSINESS-RULES.md create mode 100644 specs/issues/FIX-1496/DECISIONS.md create mode 100644 specs/issues/FIX-1496/DOCS.md create mode 100644 specs/issues/FIX-1496/PLAN.md create mode 100644 specs/issues/FIX-1496/SPEC.md create mode 100644 specs/issues/FIX-1496/figures/the-four-gaps.svg create mode 100644 specs/issues/FIX-1496/poc/gap-check/README.md create mode 100644 specs/issues/FIX-1496/poc/gap-check/check.mjs diff --git a/specs/issues/FIX-1496/BUSINESS-RULES.md b/specs/issues/FIX-1496/BUSINESS-RULES.md new file mode 100644 index 0000000000..f39dc30140 --- /dev/null +++ b/specs/issues/FIX-1496/BUSINESS-RULES.md @@ -0,0 +1,65 @@ +# FIX-1496 · Business rules + +[Spec](SPEC.md) · [Decisions](DECISIONS.md) · **Rules** · [Plan](PLAN.md) · [Docs](DOCS.md) + +The cases, written as rules. Each says what a person or the system does and what happens. The +*proved by* column is the check the plan runs. A human reviews this page; the plan turns it into +work. + +## What counts as a real artifact + +| # | When | Then | Proved by | +|---|---|---|---| +| BR-1 | The run finishes and the goal's process exits | The artifact resolves at its address with nothing of the run still alive — no held checkout, no live store | Goal, on the real path | +| BR-2 | The artifact is inspected for the lab's fingerprints | No graded token of its content appears in the lab's own code, its fixtures, or the prompt the seat was handed | Goal · the held-out discipline, inverted onto the product | +| BR-3 | The acceptance condition the brief stated is executed in the produced tree | It passes, **and** the same condition run against the base ref does not. A condition that was already true is no evidence | Goal · both halves, or the leg is vacuous | +| BR-4 | A run produces an artifact that does not satisfy the condition | The row does not settle `completed`. The artifact existing is never sufficient | Goal · control `ignores-the-brief` | +| BR-5 | A run reports it stopped at its budget, having committed something | The row does not settle `completed`, exactly as today | Existing gate · control `stopped-at-limit` | + +![Two lanes over the DevForce path showing which stages are unchanged and which four gaps close](figures/the-four-gaps.svg) + +The right-hand end of the *after* lane is this group: the artifact stage and the grading beside +it. BR-1 to BR-3 are the three properties [D1](DECISIONS.md#d1) names, one rule each. + +## The channel is driven, not decorated + +| # | When | Then | Proved by | +|---|---|---|---| +| BR-6 | An operator posts on the feature channel | The channel opens, the post lands in its transcript, and the EM seat is reached | Goal | +| BR-7 | The EM seat is reached by that post | It files exactly one row on the feature board, addressed to the `coder` assignee | Goal · one row, not one per member | +| BR-8 | The post reaches the other declared members | No board row is filed by any of them, and the reviewer seat is dispatched nothing | Goal · control `reviewer-files` | +| BR-9 | Nobody posts | No row is filed and no run happens. The board does not start itself | Goal · the negative half of BR-6 | +| BR-10 | The channel is opened without an org | Refused at the transport door, as today | Existing gate (BR-17 there), unchanged | + +## The run, and what survives it + +| # | When | Then | Proved by | +|---|---|---|---| +| BR-11 | The proof runs | It runs on stores that are on disk, and the row, the run record and the transcript are all readable from a fresh process over the same file | Goal · the store is closed and re-opened | +| BR-12 | The coder seat's run is dispatched | Its prompt carries every held-out token from the seat's own instructions, brief and skills — unchanged from today | Existing gate, and re-asserted here | +| BR-13 | The harness is unavailable or unauthenticated | The proof **fails loudly**. It never degrades to a scripted stub | Goal · no stub fallback, deliberately | +| BR-14 | The pull-request leg is not configured | The proof runs its default leg and says so in its own output. It does not silently report the credentialed claim | Goal · the two legs are named in the verdict, never conflated | +| BR-15 | The same proof is re-run a year later | The default leg needs `git`, a temporary directory and a signed-in harness. It needs no `gh`, no token and no network | Read of the check's own stated requirements | + +## What the proof must not quietly become + +| # | When | Then | Proved by | +|---|---|---|---| +| BR-16 | The change is reviewed against the epic's fences | No L1 type, no `TaskStatus` value, no second work plane, no new substrate, nothing under `packages/` | Diff gate: every changed path is inside `goals/devforce-lab/` or `specs/issues/FIX-1496/` | +| BR-17 | The existing two checks are re-run after the change | Both still pass with their existing claims intact, and their verdict logs are appended to, never rewritten | CI · both siblings run | + +## Failure taxonomy + +**Fatal, and loud:** an unavailable harness (BR-13), a tree that does not load, a base ref the +scratch repository lacks — all refused before an attempt is charged. **Non-fatal and retried:** a +run that finishes and leaves nothing re-pends the row with its reason and errors once the budget +is spent, exactly as the existing gate proves. **Never silent:** an unconfigured pull-request leg +(BR-14) degrades the *claim*, not the run, and the verdict says which leg it ran. + +## Acceptance criteria this issue owns + +One command runs the proof end to end on a machine with a signed-in harness, and the +`goals/devforce-lab/` verdict log gains a dated `PASS` row naming the artifact's address, the +branch, and which leg ran. The three properties in [D1](DECISIONS.md#d1) hold, each with the +control that makes it go red. **A passing check whose controls were never run does not close this +issue** — the red states are part of the deliverable, not a nicety. diff --git a/specs/issues/FIX-1496/DECISIONS.md b/specs/issues/FIX-1496/DECISIONS.md new file mode 100644 index 0000000000..7157e56cd3 --- /dev/null +++ b/specs/issues/FIX-1496/DECISIONS.md @@ -0,0 +1,119 @@ +# FIX-1496 · Decisions + +[Spec](SPEC.md) · **Decisions** · [Rules](BUSINESS-RULES.md) · [Plan](PLAN.md) · [Docs](DOCS.md) + +What was considered, what was chosen, why, and what each locks in. Three decisions are the +sign-off surface; everything else here is context for them. + +## The tree + +```mermaid +flowchart TD + I["FIX-1496 · which artifact counts, and how thin"] --> D1["D1 · a pull request on a separate repo
real = three falsifiable properties"] + D1 -.->|"rejected"| X1["a commit in a durable local repo
only the runner can reach it · closer to a transcript"] + D1 -.->|"rejected"| X2["a pull request on this repository
strongest evidence, but an agent opening PRs on production"] + I --> D2["D2 · grade the product against a
pre-stated, executable condition"] + D2 -.->|"rejected"| X3["keep grading only the prompt
proves the plumbing, never the work"] + I --> D3["D3 · the row is filed in answer to a
post on the feature channel"] + D3 -.->|"rejected"| X4["leave the channel declared and unused
ER-DevForce's own words go unmet"] +``` + +Solid edges are what you are signing. Dashed edges lost, and the label says why. + + +## D1 · The artifact is a pull request on a separate throwaway repository, and "real" is defined by three falsifiable properties rather than by taste + +| | | +|---|---| +| **Instead of** | A commit in a durable local repository, or a pull request against `flow-state-dev` itself | +| **Because** | ER-DevForce contrasts a real artifact with *a transcript*. A commit only the machine that produced it can reach is a transcript with a SHA. A pull request has an address a person opens, a title and a body the run wrote, and a diff a person accepts or rejects — which is the thing the launch claim is actually about | +| **Locks in** | Producing the release proof costs a credential, a network call and a throwaway repository, once. And we have committed to a definition of *real work* that the next Lab proof has to keep meeting — a later artifact that cannot satisfy the three properties does not get to call itself real by analogy | + +**The three properties, each with its falsifier.** This is what makes *"would a person accept +this as real work?"* checkable instead of a matter of taste: + +| Property | Falsified by | +|---|---| +| **It outlives the run at an address a third party resolves** | The goal's process has exited and its temporary directories are gone; fetching the address returns nothing | +| **The Lab did not author it** | Any graded token of the artifact's content is also found in the lab's own code, its fixtures, or the prompt — the held-out discipline, inverted onto the product | +| **It satisfies a condition the requester stated before the run, executed** | The condition is run in the produced tree and does not pass; or the condition would have passed before the run, which makes it no evidence at all | + +**What would change my mind:** if the owner reads "exists outside the run" as *outside the +process* rather than *outside the machine*, a bare repository at a declared path is cheaper and +needs no credential. Say so and the pull-request leg becomes optional rather than the proof. + + +## D2 · The proof grades what the run produced, against an acceptance condition the brief states first and a machine executes + +| | | +|---|---| +| **Instead of** | Keeping the existing rule that only the prompt is graded and the product never is | +| **Because** | The existing rule is right about what it guards — a model writes a plausible file without reading anything, so the file is no evidence a document was *read*. But ER-DevForce asks for a **work product**, and nothing that refuses to look at the product can prove one. The way out is not to grade the file's contents by eye; it is to grade it against a condition the *requester* wrote down before the run and a machine can execute | +| **Locks in** | The feature brief stops being flavour text and becomes a contract. Every future DevForce brief owes an executable done-condition, and a Lab task that cannot state one cannot be proved this way — which is a real limit on what this proof shape can cover | + +Both gradings stay, because they answer different questions. *Did the seat's own files reach the +run?* is the prompt's job and is unchanged. *Did the run produce work someone asked for?* is the +condition's job and is new. A run that improvised without reading the brief fails the second +while possibly passing neither. + + +## D3 · The row is filed in answer to a post on the feature channel, not by calling the EM seat's action directly + +| | | +|---|---| +| **Instead of** | Leaving the declared channel walked-but-unopened, as it is today, and filing through the EM seat's own action | +| **Because** | ER-DevForce says *seats **and channels** used honestly*. Today the channel is a Markdown file the loader reads and nothing opens — honest by the letter, empty in fact. The post-reaches-a-seat path is already proven in two sibling labs on the same primitive, so closing this costs composition, not invention | +| **Locks in** | The proof now depends on the channel fan-out as well as the board hand-off, so a regression in either fails it. That is the point: it is one path end to end, and a proof that skipped the front door would have been reporting on a shorter path than the one we ship | + +**What would change my mind:** if [ER-Collab](../../epics/FIX-1457/BUSINESS-RULES.md#er-collab)'s +producer is filed this cycle and takes the channel leg explicitly, this becomes duplicated +evidence and should be dropped here to keep the path thinnest. Nothing has been built against it. + +## Decided, not asked + +- **This is a delta on `goals/devforce-lab/`, not a new Lab tree.** Roughly four fifths of the + path is built and passing; the issue's own fence says compose + ([ER-11](../../epics/FIX-1457/BUSINESS-RULES.md), [ER-25](../../epics/FIX-1457/BUSINESS-RULES.md)). +- **The new proof is a third sibling goal**, not an edit to either existing check. They carry + claims and verdict logs that are still true; rewriting one to mean something else would + invalidate evidence for no gain. +- **The board stays declared in code.** Moving it onto the channel's `boards:` frontmatter would + mint a second board and change which one the cross-flow claim gate compares against. The + interim tax is already labelled as one; paying it for one more proof is cheaper than the risk. +- **No package under `packages/` changes.** Everything the proof needs — the channel primitives, + the harness manager, the `gh` completion probe — already ships. +- **The existing never-run commit check gets run too.** It is the cheaper half of the same + wiring, and leaving a written-but-unrun check beside a newly passing one is how the next reader + learns that unrun checks are normal here. +- **Stores go on disk for the new proof only.** ER-3 asks for durable across the run; the + model-free gate is a contract check and has no such requirement. + +## Considered and dropped + +| Alternative | Why not | +|---|---| +| **A commit in a durable local bare repository** | Cheapest, needs no credential — and fails the first property. An artifact only the producing machine can reach is the same defect as the temp directory, arriving more slowly | +| **A pull request against `flow-state-dev` itself** | The strongest possible evidence: a human would merge it or not. Also an agent opening pull requests on the production repository, on a schedule nobody asked for, and a proof whose runtime includes a human review round | +| **Extend `it-commits-from-the-seats-own-file` instead of adding a sibling** | Thinner by one file, and it rewrites a check whose stated claim is deliberately *a commit, not a pull request*. Its verdict log would then describe a check that no longer exists | +| **Grade the artifact by reading its contents for tokens** | The failure the existing goal exists to refuse. A model that never read the brief can still emit a token it was told to emit | +| **Move the board onto the channel's `boards:` declaration** | Removes the interim tax and is the authored shape FIX-1385 settled — and changes the board the hand-off gates against. A substrate-shaped change wearing a polish label is exactly [ER-25](../../epics/FIX-1457/BUSINESS-RULES.md) | +| **Wait for [FIX-1440](https://linear.app/fixpoint-labs/issue/FIX-1440) to land first** | Checked, not assumed: the lab reads dispatch parentage only as provenance, which FIX-1440's own owner amendment keeps, and reads nothing from the browsable child-session surface it removes. Re-derived by [`poc/gap-check/`](poc/gap-check/README.md) | + +## Settled + +- **FIX-1440 does not fence this work** — re-derived by + [`poc/gap-check/`](poc/gap-check/README.md), claim 5: nothing under `goals/devforce-lab/` + reads the browsable child-session surface FIX-1440 removes, and the one parentage read is the + provenance form the owner amendment explicitly retains. FIX-1440 is `Spec Approved`, not + shipped; that remains true and remains irrelevant here. +- **The gap table is four rows, not more** — [`poc/gap-check/`](poc/gap-check/README.md) asserts + all twelve facts the spec rests on, including a totality assertion over every TypeScript file + in the lab, and goes red under four planted defects. Run 2026-09-22: `PASS — 12 claims held`. + +## How it got here + +- **Draft** — framed as *the proof is written and has never been run*, after finding + `it-commits-from-the-seats-own-file` complete, type-checked and logged `NOT RUN`; the approach + is a four-gap delta on the existing lab rather than a new DevForce path, and the artifact + question the Architect left open is answered as a pull request with three falsifiable + properties. diff --git a/specs/issues/FIX-1496/DOCS.md b/specs/issues/FIX-1496/DOCS.md new file mode 100644 index 0000000000..9e77bc0125 --- /dev/null +++ b/specs/issues/FIX-1496/DOCS.md @@ -0,0 +1,76 @@ +# FIX-1496 · Documentation draft + +[Spec](SPEC.md) · [Decisions](DECISIONS.md) · [Rules](BUSINESS-RULES.md) · [Plan](PLAN.md) · **Docs** + +## No published-documentation impact, and why + +Nothing user-facing changes. This issue adds no public API, no configuration, no observable +framework behaviour and no new concept a reader would search for: every changed path is inside +`goals/devforce-lab/`, which is retained evidence rather than a shipped surface, and no package +under `packages/` is touched (enforced as [BR-16](BUSINESS-RULES.md), checked by `V9`). + +Checked rather than assumed, against the surfaces the docs-scoping heuristics name: + +| Surface | Verdict | +|---|---| +| `apps/docs/docs/` (reference) | No change. No new primitive, no altered semantics. The channel, board and harness-manager primitives this composes are documented already and behave identically | +| `apps/docs/guides/` | No change. This unlocks no new end-to-end workflow for a framework user — it proves one we already describe | +| `packages/*/README.md` | No change. No public export changes | +| `apps/docs/blog/` | No change. A launch-readiness proof is internal evidence; announcing it would be announcing a QA result | +| `docs/architecture/*` | No change. No locked contract moves | +| Changeset (BP-022) | **None required.** No published package changes, so no downstream consumer has anything to learn. Noted in the PR rather than as an empty fragment | + +## The prose that does change, and where + +Two internal documents, both of which ship in the same PR as the code they describe. + +### UPDATE · `goals/devforce-lab/lab/README.md` · *What this directory owns, and why each piece is here* + +Add two rows to the existing table and one line of framing above it. Draft: + +> The lab now opens the channel it declares. `CHANNEL.md` used to be walked and never driven — +> the positive half of "an unwalked folder loads as nothing" and nothing more. The feature +> channel is now the front door: a post on it is what reaches the EM seat, and the EM seat is +> what files the row. The board hand-off underneath is unchanged. + +| File | Why it is the lab's | +|---|---| +| the channel door in `host.mts` | The org is not threaded through `openChannels` yet ([FIX-1412](https://linear.app/fixpoint-labs/issue/FIX-1412)), so the session client is wrapped to inject one. The same wrap the two sibling labs carry, for the same reason | +| the two artifact legs in `scratch-repo.mts` | Whether the artifact leaves the machine is an operator's choice, not the lab's. The default leg stays a temporary repository so the check is re-runnable with no credential; the credentialed leg gives the artifact an address a person can open | + +### UPDATE · `goals/devforce-lab/lab/README.md` · *What it works around* + +Append one paragraph. Draft: + +> **The pull-request leg needs a credential, and the check says which leg it ran.** A proof that +> quietly ran the weaker leg and reported the stronger one would be worse than no proof, so the +> leg is named in the verdict rather than inferred from whether `gh` happened to be installed. + +### CREATE · `goals/devforce-lab/it-ships-an-artifact-a-person-can-open/goal.md` + +The new check's own document, in the shape its two siblings already use — outcome, input, signal, +anti-game, controls table, model, run command, what it establishes and what it does not, verdict +log. The implementer writes it against what the check actually asserts; two sections are fixed by +this spec and are drafted here. + +**Outcome** (draft): + +> Somebody posts one line into a channel a Markdown file declared. A coordinator seat reads the +> post and files one row. A working seat the row names wakes in a checkout of its own, runs a +> real coding agent from a prompt built out of its own files, and opens a pull request — at an +> address that still resolves when the run is over, carrying work that satisfies a condition the +> brief stated before the run started. + +**What this establishes, and what it does not** (draft): + +> It establishes that a hired Workforce can take a request through a channel and hand back a work +> product a person can open and judge. It does **not** establish that the product is *good* — +> only that it satisfies the acceptance condition its requester wrote down, which is the strongest +> claim a machine can make here and is deliberately weaker than a human review. It does not +> establish anything about a second Lab, about CyberForce, or about unbounded product delivery. + +## Publication ownership + +This issue publishes both documents in its own PR. The epic's shared narrative — what Workforce +is and how the three exit proofs relate — belongs to +[FIX-1457's wrap-time docs pass](../../epics/FIX-1457/DOCS.md), and is not duplicated here. diff --git a/specs/issues/FIX-1496/PLAN.md b/specs/issues/FIX-1496/PLAN.md new file mode 100644 index 0000000000..c979779414 --- /dev/null +++ b/specs/issues/FIX-1496/PLAN.md @@ -0,0 +1,141 @@ +# FIX-1496 · Plan + +[Spec](SPEC.md) · [Decisions](DECISIONS.md) · [Rules](BUSINESS-RULES.md) · **Plan** · [Docs](DOCS.md) + +Written for the implementing agent. IDs cross-reference [BUSINESS-RULES.md](BUSINESS-RULES.md) +(BR-n) and [DECISIONS.md](DECISIONS.md) (D-n). `tdd`. One PR. + +## Surfaces + +| ID | Package · role | Change | Rules | +|---|---|---|---| +| S1 | `goals/devforce-lab/lab` · the authored brief | Rewrite the feature brief around an **executable** done-condition: an exported greeting function plus a test the repository's own runner picks up, and the existing held-out marker carried into the pull request body. Keep the marker — the prompt is still graded for it (D2) | BR-3 BR-12 | +| S2 | `goals/devforce-lab/lab` · the host | Open the declared feature channel (`channelInstances` + `openChannels`, as both sibling labs do) and expose posting on it. The EM seat files its row in answer to the post rather than through a direct action call (D3) | BR-6 BR-7 BR-8 BR-9 BR-10 | +| S3 | `goals/devforce-lab/lab` · the artifact's repository | Give the scratch repository a **real** remote and a second construction mode: today's temp-dir repository for the default leg, and a configured remote for the credentialed leg. Name the leg in what the helper returns so the verdict can report it (D1) | BR-1 BR-14 BR-15 | +| S4 | `goals/devforce-lab/lab` · the phase | Make the done-condition a slot rather than a fixed commit probe. Default stays *a commit the base ref lacks*; the credentialed leg uses the `gh` completion probe `labs/conductor` already ships. **Do not re-implement it** | BR-1 BR-4 BR-5 | +| S5 | `goals/devforce-lab/lab` · the acceptance gate | Execute the brief's condition in the produced tree, and the same condition against the base ref. Both halves, or the leg proves nothing (D2) | BR-3 BR-4 | +| S6 | `goals/devforce-lab/it-ships-an-artifact-a-person-can-open/` · **new sibling goal** | The runner, its `goal.md` (outcome · input · signal · anti-game · controls · verdict log), and the controls below. Reuses `openLab` with the S2–S5 options; **does not edit either existing check** | all | +| S7 | `goals/devforce-lab/lab/README.md` | Extend *What this directory owns* and *What it works around* for the channel door and the two legs. Record that the artifact now leaves the run | — | +| S8 | The verdict logs | Run the new proof **and** the never-run `it-commits-from-the-seats-own-file`, and append a dated row to each. Appending only — never rewrite an existing row | BR-17 | + +**Nothing is removed.** Named explicitly because tenet 3 expects the question asked: the two +existing checks keep their claims, the board stays declared in code, and the `coder` kind's +second board declaration stays the labelled interim tax it already is +([D · Decided, not asked](DECISIONS.md#decided-not-asked)). + +## Sequence + +```mermaid +flowchart TD + S1["S1 · brief gains an executable condition"] --> S5["S5 · the acceptance gate"] + S2["S2 · open and drive the channel"] --> S6["S6 · the new sibling goal"] + S3["S3 · the artifact's repository, two legs"] --> S4["S4 · done-condition as a slot"] + S4 --> S6 + S5 --> S6 + S6 --> S8["S8 · run both, record both verdicts"] + S6 --> S7["S7 · the lab README"] +``` + +One PR. S1–S5 are independent of each other and all feed S6; S8 is the proof itself and is last +because it is the only step that needs a credentialed machine. + +## Checks + +| ID | Runs after | Passes when | +|---|---|---| +| V1 | S2 | BR-6, BR-7, BR-9. A post opens the channel, lands in the transcript, and produces **exactly one** row. No post, no row | +| V2 | S2 | BR-8. Control `reviewer-files`: the reviewer seat is made to answer the post by filing; the check goes red on the dispatch record, not merely on the transcript | +| V3 | S3, S4 | BR-14, BR-15. The default leg runs with no `gh`, no token and no network, and the verdict names which leg ran. Both legs exercised | +| V4 | S5 | BR-3, **both halves**: the condition passes in the produced tree and fails against the base ref. Control `already-passing`: a condition true before the run goes red | +| V5 | S5 | BR-4. Control `ignores-the-brief`: a run that commits something unrelated does not settle the row `completed` | +| V6 | S6 | BR-2. Every graded token of the artifact's content is absent from the lab's `.mts` files, its fixtures and the prompt — asserted **before** any verdict is read | +| V7 | S6 | BR-11. The store is closed and a fresh process reads the same row, run record and transcript | +| V8 | S6 | BR-13. Control `no-harness`: the check fails loudly and does not fall back to a stub | +| VG | S8 | **The goal.** One command on a credentialed machine: a post produces a pull request whose address resolves after the process exits, whose content the lab did not author, and which satisfies the brief's condition. `goals/devforce-lab/it-ships-an-artifact-a-person-can-open/run.mts` | +| V9 | S8 | BR-16. Diff gate: every changed path is under `goals/devforce-lab/` or `specs/issues/FIX-1496/`. Nothing under `packages/` | +| V10 | S8 | BR-17. Both existing siblings still pass, claims unchanged | + +**Every control must be seen red.** A green check nobody has watched fail is not evidence +(tenet 7), and the existing labs record each control's red state in the verdict log — match that. + +## Pinned names · the only two + +| Where | Name | Why pinned | +|---|---|---| +| The new goal directory | `goals/devforce-lab/it-ships-an-artifact-a-person-can-open` | It is the claim, and the verdict log is cited from the epic | +| The brief's held-out marker | `FEATURE-BRIEF-E61B8` | The existing prompt grading keys on it; changing it silently weakens a passing check | + +Everything else is yours to name. + +## Guardrails + +| Rule | Because | +|---|---| +| The artifact's content is graded **only** against the brief's condition, never for tokens | A model emits a token it was told to emit. The condition is the only grading that a run which never read the brief cannot fake (D2) | +| BR-3 runs both halves — the condition against the produced tree *and* against the base ref | A condition that was already true reports PASS on a run that did nothing. This is the vacuous-green shape the sibling labs each found the hard way | +| The credentialed leg and the default leg are named in the verdict, never merged into one claim | A proof that quietly ran the weaker leg and reported the stronger one is worse than no proof (BR-14) | +| No stub fallback when the harness is missing | A model-backed check that degrades to a scripted run is the exact failure its model-free sibling exists to detect. The existing check already states this; keep it | +| Every changed path stays inside `goals/devforce-lab/` or this spec | [ER-11](../../epics/FIX-1457/BUSINESS-RULES.md) and [ER-25](../../epics/FIX-1457/BUSINESS-RULES.md). A package change here is a substrate epic wearing a polish label, and V9 is the fence | +| Compose `openChannels` and conductor's `gh` probe; do not re-implement either | [ER-4](../../epics/FIX-1457/BUSINESS-RULES.md). A second copy of a proven path is a second thing to keep true | + +## Docs + +Reconcile and publish [DOCS.md](DOCS.md) after VG passes. No `apps/docs` page and no package +README changes: nothing user-facing moves. The changed prose is the lab's own README and the new +goal's `goal.md`, both internal evidence. + +## Sketch · pseudocode, illustrative, react to the shape + +``` +open the lab, as today, plus: + stores ← on disk, under a path this check made (BR-11) + channels ← open the declared feature channel (D3) + repository ← default: a temp repo, as today + credentialed: a repo with a real remote (D1) + doneCondition ← default: a commit the base ref lacks, as today + credentialed: conductor's gh probe (D1) + +the proof: + post on the feature channel (BR-6) + wait for one row to appear on the feature board (BR-7) + wait for the row to settle + assert the artifact resolves with the run's process gone (BR-1) + assert none of its graded content lives in the lab (BR-2) + run the brief's condition in the produced tree → passes (BR-3) + run the brief's condition against the base ref → fails (BR-3) +``` + +**POC:** [`poc/gap-check/`](poc/gap-check/README.md) — a Node script that re-derives the twelve +facts this spec rests on straight from the repository, with a totality assertion over every +TypeScript file in the lab and four planted defects each watched going red. Run +`node specs/issues/FIX-1496/poc/gap-check/check.mjs`. **The premise held**: all twelve green on +2026-09-22, including the one the brief flagged as worth checking — FIX-1440 does not fence this +work. Nothing in the design changed because of it; it is here so a reviewer can re-run the +argument instead of re-reading it. + +## At implement time + +- **Confirm how a channel post reaches a seat's action.** The plan names the seam, not the + entry. `goals/pentest-lab/lab/host.mts` and `goals/manager-queue-lab/lab/host.mts` both do + this; copy whichever is closer. The fan-out reaches *every* declared member, so decide there + what the coder and reviewer seats do with a post — BR-8 only requires that neither files. +- **The org is not threaded through `openChannels`** (FIX-1412). Both sibling labs wrap the + session client to inject it. Wrap, do not work around, and do not fix FIX-1412 here. +- **Check whether [FIX-1440](https://linear.app/fixpoint-labs/issue/FIX-1440) has landed.** It + was `Spec Approved` and unshipped when this was written, and + [`poc/gap-check/`](poc/gap-check/README.md) claim 5 says it does not fence this work. Re-run + the checker rather than re-reading this sentence. +- **Check whether [FIX-1481](https://linear.app/fixpoint-labs/issue/FIX-1481) has landed.** If + the org-level inventory view exists by then, note in the verdict that the run is + Devtool-visible. Soft; not a gate. +- **`gh` is not installed on every runner.** It was absent from the machine this spec was + written on. Conductor's probe already fails with an instruction rather than a stack trace; + make sure the default leg is what runs when it is missing. + +## Follow-ups + +- `labs/conductor`'s own `implement-phase-opens-a-pr` goal has **no verdict-log rows at all** — + so "conductor proves the `gh` probe" is a claim about a code path, not a recorded run. Out of + scope here; worth a ticket against the conductor lab. +- The `coder` kind declaring the feature board a second time is an interim L1 tax carved onto + [FIX-1408](https://linear.app/fixpoint-labs/issue/FIX-1408). Unchanged by this issue, still owed. diff --git a/specs/issues/FIX-1496/SPEC.md b/specs/issues/FIX-1496/SPEC.md new file mode 100644 index 0000000000..4d31d2f6ad --- /dev/null +++ b/specs/issues/FIX-1496/SPEC.md @@ -0,0 +1,103 @@ +# FIX-1496 · ER-DevForce proof: thinnest DevForce path ships a real artifact + +**Spec** · [Decisions](DECISIONS.md) · [Rules](BUSINESS-RULES.md) · [Plan](PLAN.md) · [Docs](DOCS.md) + +Feature · `goals/devforce-lab` only · **small** · 1 PR · epic [FIX-1457](../../epics/FIX-1457/SPEC.md) ([ER-DevForce](../../epics/FIX-1457/BUSINESS-RULES.md#er-devforce)) + +## Five people, before and after + +| Someone who… | Today | After | +|---|---|---| +| **is deciding whether Workforce is ready to ship** | "A Lab can build" rests on a check that was written, type-checked, and **never executed**. Its verdict log has one row and that row reads `NOT RUN` | Opens a pull request a hired coder seat wrote, and reads the diff. The verdict log carries a dated `PASS` with the pull request's address in it | +| **asks what the workforce actually produced** | A commit in a temporary directory on whatever machine ran it, swept away with the rest of `/tmp` | A pull request with a title, a body and a diff, at an address that still resolves after the run is over | +| **asks whether the work is any good, not merely that it exists** | Nothing grades the product. The check grades the *prompt*, deliberately, so a model that improvised a plausible file still passes | The brief names an acceptance condition **before** the run, and the check executes it against what the run produced. Improvising does not pass | +| **asks whether the seats really used their channels** | The feature channel is a Markdown file the loader walks and nothing ever opens | A post on the feature channel is what starts the work. The channel is driven, not decorated | +| **has to re-run this proof a year from now** | Impossible: it has never been run once | One command. The credentialed pull-request leg is a flag; the default leg needs no network, no token and no `gh` | + +**Why now.** W5's done condition is three exit proofs and none has been run +([FIX-1457](../../epics/FIX-1457/SPEC.md)). ER-DevForce is the one that decides whether +"Workforce can host a Lab that builds" is a claim or a fact. The research finding is that it is +**nearly built already** — and that the last mile is not more substrate, it is running the thing +and letting what it makes survive. + +## What changes + +![Two lanes over the same five-stage DevForce path. Today three stages are built and green while the declared channel sits unused and the artifact is a commit in a temp directory; a banner reads never run, on in-memory stores. After, the same five stages with four marked gaps closed: a post on the feature channel drives the filing, the run is on durable on-disk stores, the artifact is a pull request at an address a person can open, and it is graded against an acceptance condition the brief stated before the run.](figures/the-four-gaps.svg) + +The five stages are the same in both lanes, and the middle three are untouched. Only the two +ends move, plus two properties of the run. **Nothing new is built** — the four gaps are the whole +issue ([D1](DECISIONS.md#d1), [D2](DECISIONS.md#d2), [D3](DECISIONS.md#d3)). + +**The brief the coder seat works from, as its author writes it** — this is the surface that turns +"a file exists" into "work a person would accept": + +```diff + # Feature brief + + Add a greeting module to the repository you are working in. + +- Write the file `GREETING.md`, and make its first line exactly: ++ Export a `greet(name)` function that returns a greeting, and cover it with a ++ test the repository's own test runner picks up. ++ ++ ## Done when ++ ++ The repository's test run reports at least one passing test and no failing ++ ones. Before your change it reports none, so an untested module is not done. ++ ++ Put this marker in the body of the pull request you open: + + FEATURE-BRIEF-E61B8 +``` + +The marker stays because the existing check grades the *prompt* for it, which is how we know the +seat read its own files. What is new is everything above it: a condition a machine can run. + +## How the proof reaches an artifact + +```mermaid +flowchart LR + P["an operator posts on the feature channel"] --> E["the EM seat · files one row"] + E --> B["the feature board · one row, one assignee"] + B -->|"cross-flow hand-off"| C["the coder seat · its own checkout"] + C --> H["a real coding agent · prompt built from the seat's own files"] + H --> A["a pull request · pushed, opened, addressable"] + A --> V["the acceptance condition the brief stated, executed"] + R["the reviewer seat · declared, never reached"] -.->|"still never dispatched"| B +``` + +Everything from the EM seat to the coding agent exists and passes today under a scripted stub. +The two ends are what this issue adds, and the dashed edge is the negative claim the existing +gate already holds. + +## What stays as it is + +- **The substrate.** Boards, inventory, manager-queue, org identity. Composed, never re-decided + ([ER-4](../../epics/FIX-1457/BUSINESS-RULES.md)). No package under `packages/` changes. +- **The existing two checks.** `it-wakes-the-seat-a-file-declared` (the model-free contract gate) + and `it-commits-from-the-seats-own-file` (the commit claim) keep their claims and their verdict + logs. The new proof is a **third sibling**, not a rewrite of either. +- **The board's provenance.** It stays declared in code, with the `coder` kind's second + declaration as the labelled interim tax it already is. Moving it onto the channel would change + which board the hand-off gates against, and that is not this issue's risk to take. +- **Devtool.** Observing this run in the inspector is [ER-Devtool](../../epics/FIX-1457/BUSINESS-RULES.md#er-devtool)'s + job and [FIX-1481](https://linear.app/fixpoint-labs/issue/FIX-1481) owns it. Soft, not a gate here. +- **CyberForce, kitchen-sink, and any Lab product surface.** Out + ([ER-11](../../epics/FIX-1457/BUSINESS-RULES.md), [ER-24](../../epics/FIX-1457/BUSINESS-RULES.md#er-24)). + +## Sign off + +1. **[D1](DECISIONS.md#d1) · The artifact is a pull request on a separate throwaway repository, + and "real" is three falsifiable properties rather than a judgement call.** *If wrong:* the + release proof needs a token and network once, and we have committed to a definition of "real + work" that a later Lab has to keep meeting. +2. **[D2](DECISIONS.md#d2) · The proof grades what the run produced, against a condition the + brief stated first and a machine executes.** *If wrong:* every future DevForce brief owes an + executable acceptance condition, and a brief that cannot state one cannot be proved this way. +3. **[D3](DECISIONS.md#d3) · The row is filed in answer to a post on the feature channel, not by + calling the EM seat's action directly.** *If wrong:* one extra leg in a proof whose whole + virtue is thinness, for a claim ER-Collab may cover anyway. + +**Open: none.** Number 1 is the one to weigh — it is the question the Architect deliberately left +open, answered here as a proposal with a falsifier rather than left open again. The reasoning and +what lost: [DECISIONS.md](DECISIONS.md). The cases: [BUSINESS-RULES.md](BUSINESS-RULES.md). diff --git a/specs/issues/FIX-1496/figures/the-four-gaps.svg b/specs/issues/FIX-1496/figures/the-four-gaps.svg new file mode 100644 index 0000000000..bf784dbbf0 --- /dev/null +++ b/specs/issues/FIX-1496/figures/the-four-gaps.svg @@ -0,0 +1,101 @@ + + + + + THE CHANNEL + THE EM SEAT + THE BOARD ROW + THE CODER SEAT + THE ARTIFACT + + + + TODAY + + + declared, unused + CHANNEL.md is walked + and never opened + + + files the row + through its own action, + not through the channel + + + crosses flows + reaches the seat the + assignee names + + + a real agent runs + Claude Code in the slot, + prompt from its own files + + + a commit, then gone + in an OS temp directory + nobody can open later + + + + NEVER RUN — the model-backed check is written and type-checked; its verdict log holds one row and that row says NOT RUN. + And the run is on in-memory stores, which ER-3 does not accept as durable. + + + + AFTER + + + a post drives it + GAP 1 · the channel is + opened and used + + + files the row + unchanged — it answers + the post instead + + + crosses flows + unchanged + + + a real agent runs + GAP 2 · on durable + on-disk stores + + + a pull request + GAP 3 · an address a + person opens · GAP 4 graded + + + + + + + Nothing new is built. The middle three stages are unchanged; the four marked gaps are what this issue closes. + diff --git a/specs/issues/FIX-1496/poc/gap-check/README.md b/specs/issues/FIX-1496/poc/gap-check/README.md new file mode 100644 index 0000000000..22c60e0214 --- /dev/null +++ b/specs/issues/FIX-1496/poc/gap-check/README.md @@ -0,0 +1,56 @@ +# `gap-check` — the spec's factual base, re-derived from the repository + +An experiment, not a supported API. Nothing imports it, it is not a workspace package, and it +is in no build, test or lint discovery path. It exists so the gap table in +[`SPEC.md`](../../SPEC.md) can be checked rather than believed. + +## What it checks + +FIX-1496's argument is that ER-DevForce is mostly built and exactly four things are missing. +That is a counted claim, and a counted claim argued in review converges one row at a time while +the rows nobody looked at stay wrong. So the twelve statements the spec rests on are asserted +here instead: + +| Claim | The spec row it holds up | +|---|---| +| Every TypeScript file under `goals/devforce-lab/` is classified | Nothing in the lab escaped the claims below | +| Two goal checks, exactly one model-backed, no PASS recorded on it | "The proof is written and has never been run" | +| The artifact's repository is created under the OS temp directory, and nothing pushes it | "The artifact does not outlive the run" | +| The model-backed check runs on `inMemoryStores()` | ER-3's *durable across the run* is unmet | +| Nothing in the lab calls `openChannels`, one `CHANNEL.md` is declared, and **both** sibling labs do call it | "The channel is walked but never driven, and the path is proven next door" | +| Nothing reads the browsable child-session surface; parentage is read as provenance only | FIX-1440 does not fence this work | + +## Run it + +```bash +node specs/issues/FIX-1496/poc/gap-check/check.mjs +``` + +Node only. No install, no network, no credential, no model. + +## The two properties a hand-written checker usually lacks + +**A totality assertion.** Claim 0 asserts that *every* `.mts`/`.ts` file under the lab is +classified as a goal runner, lab root code, or a kind. A checker that inspects only the files it +already knew about cannot report the one nobody listed, so this one runs first and is fatal. + +**Negative controls, actually run.** `--plant ` injects the exact defect a claim exists to +catch. `--plant list` prints them. Observed 2026-09-22 on `spec/FIX-1496`: + +| Plant | Result | +|---|---| +| *(none)* | **PASS** — 12 claims held | +| `unclassified-file` | **FAIL** on claim 0 alone, naming the planted path, and stops before the rest | +| `channel-is-opened` | **FAIL** on the channel gap alone; the other 11 hold | +| `durable-stores` | **FAIL** on the durability gap alone; the other 11 hold | +| `model-check-passed` | **FAIL** on "never run" alone; the other 11 hold | + +Each control perturbs one thing and goes red on one claim, so a red here names its own cause. +Plants are applied to the in-memory copy of the tree; nothing on disk is touched, so a control +run cannot leave the repository perturbed. + +## Limits + +It reads the repository, not a running workforce. It can tell you the channel is never opened; +it cannot tell you the proof would pass once it is. That is the goal check's job, and building +it is what this issue is for. diff --git a/specs/issues/FIX-1496/poc/gap-check/check.mjs b/specs/issues/FIX-1496/poc/gap-check/check.mjs new file mode 100644 index 0000000000..d3e4e7c722 --- /dev/null +++ b/specs/issues/FIX-1496/poc/gap-check/check.mjs @@ -0,0 +1,264 @@ +#!/usr/bin/env node +/** + * Re-derive FIX-1496's factual base from the repository, rather than trusting + * the spec's prose. + * + * The spec's whole argument is a gap table: ER-DevForce is mostly built, and + * exactly four things are missing. A hand-counted table like that does not + * converge by being argued about in review — each round corrects one row and + * leaves the ones nobody looked at. So it gets a checker, and the checker gets + * the two properties a hand-written one always lacks. + * + * **A totality assertion.** Every `.mts` file under `goals/devforce-lab/` is + * classified as lab code, a goal runner, or a fixture. A checker that only + * inspects the files it already knew about cannot report the one nobody listed. + * + * **A negative control.** `--plant ` injects the exact defect a claim + * exists to catch, so the green can be watched going red before it is believed. + * + * Throwaway evidence, not production code. Nothing imports it, it is not a + * workspace package, and it is not in any build, test or lint discovery path. + * + * Run: node specs/issues/FIX-1496/poc/gap-check/check.mjs + * Control: node specs/issues/FIX-1496/poc/gap-check/check.mjs --plant unclassified-file + * node specs/issues/FIX-1496/poc/gap-check/check.mjs --plant channel-is-opened + * node specs/issues/FIX-1496/poc/gap-check/check.mjs --plant durable-stores + * node specs/issues/FIX-1496/poc/gap-check/check.mjs --plant model-check-passed + * node specs/issues/FIX-1496/poc/gap-check/check.mjs --plant list + */ +import { readFileSync, readdirSync, statSync } from "node:fs"; +import { join, relative } from "node:path"; +import { fileURLToPath } from "node:url"; + +const REPO = fileURLToPath(new URL("../../../../../", import.meta.url)); +const LAB = join(REPO, "goals/devforce-lab"); +const SIBLINGS = ["goals/pentest-lab", "goals/manager-queue-lab"]; + +const PLANTS = [ + "unclassified-file", + "channel-is-opened", + "durable-stores", + "model-check-passed", +]; + +const plant = (() => { + const i = process.argv.indexOf("--plant"); + if (i === -1) return undefined; + const value = process.argv[i + 1]; + if (value === "list") { + console.log(PLANTS.join("\n")); + process.exit(0); + } + if (!PLANTS.includes(value)) { + console.error(`unknown plant "${value}"; one of: ${PLANTS.join(", ")}`); + process.exit(2); + } + return value; +})(); + +/** Every file under a directory, recursively, as [repo-relative path, contents]. */ +function filesUnder(root) { + const out = []; + for (const entry of readdirSync(root)) { + const full = join(root, entry); + if (statSync(full).isDirectory()) out.push(...filesUnder(full)); + else out.push([relative(REPO, full), readFileSync(full, "utf8")]); + } + return out; +} + +const failures = []; +const evidence = []; +const claim = (ok, red, green) => (ok ? evidence.push(green) : failures.push(red)); + +// The tree, with the plant applied to the in-memory copy only. Nothing on disk +// is touched, so a control run cannot leave the repository perturbed. +let tree = filesUnder(LAB); +if (plant === "unclassified-file") { + tree = [...tree, ["goals/devforce-lab/lab/workforce/flows/helpers.ts", "// planted\n"]]; +} +if (plant === "channel-is-opened") { + tree = tree.map(([p, t]) => + p.endsWith("lab/host.mts") ? [p, `${t}\n// planted: await openChannels(channels, {});\n`] : [p, t], + ); +} +if (plant === "durable-stores") { + tree = tree.map(([p, t]) => + p.includes("it-commits-from-the-seats-own-file") + ? [p, t.replace(/inMemoryStores\(\)/g, "sqliteStores()")] + : [p, t], + ); +} +if (plant === "model-check-passed") { + tree = tree.map(([p, t]) => + p.endsWith("it-commits-from-the-seats-own-file/goal.md") + ? [p, t.replace("| NOT RUN |", "| PASS |")] + : [p, t], + ); +} + +const text = (suffix) => tree.filter(([p]) => p.endsWith(suffix)); +const body = (suffix) => text(suffix).map(([, t]) => t).join("\n"); + +// --------------------------------------------------------------------------- +// Claim 0 — totality. Every .mts under the lab is one of three known kinds. +// +// This runs first and its failure is fatal: every claim below reads a subset of +// these files, and a file none of them classified is a file none of them saw. +// --------------------------------------------------------------------------- +{ + const mts = tree.filter(([p]) => p.endsWith(".mts") || p.endsWith(".ts")); + const unclassified = mts.filter(([p]) => { + const isRunner = /goals\/devforce-lab\/[^/]+\/run\.mts$/.test(p); + const isLabRoot = /goals\/devforce-lab\/lab\/[^/]+\.mts$/.test(p); + const isLabKind = /goals\/devforce-lab\/lab\/workforce\/flows\/workers\/[^/]+\.mts$/.test(p); + return !isRunner && !isLabRoot && !isLabKind; + }); + claim( + unclassified.length === 0, + `${unclassified.length} TypeScript file(s) under the lab are classified by no rule, so no ` + + `claim below inspected them: ${unclassified.map(([p]) => p).join(", ")}`, + `all ${mts.length} TypeScript files under the lab are classified (runners, lab root, kinds)`, + ); + if (unclassified.length > 0) { + report(); + process.exit(1); + } +} + +// --------------------------------------------------------------------------- +// Claim 1 — two goal checks, exactly one of them model-backed, and the +// model-backed one has never recorded a passing run. +// --------------------------------------------------------------------------- +{ + const goals = tree.filter(([p]) => p.endsWith("/goal.md")).map(([p]) => p); + claim( + goals.length === 2, + `the lab holds ${goals.length} goal checks, not the 2 the spec counts: ${goals.join(", ")}`, + `the lab holds exactly 2 goal checks`, + ); + + const modelBacked = tree.filter(([, t]) => /^\*\*Model:\*\*\s*real/m.test(t)); + claim( + modelBacked.length === 1, + `${modelBacked.length} goal checks declare a real model, not 1`, + `exactly 1 goal check declares a real model (${modelBacked[0]?.[0] ?? "-"})`, + ); + + const honesty = tree.find(([p]) => p.endsWith("it-commits-from-the-seats-own-file/goal.md")); + const verdicts = (honesty?.[1] ?? "") + .split("\n") + .filter((line) => /^\|\s*\d{4}-\d{2}-\d{2}/.test(line)); + const passes = verdicts.filter((line) => /\|\s*PASS/.test(line)); + claim( + verdicts.length > 0 && passes.length === 0, + `the model-backed check records ${passes.length} passing verdict(s); the spec's whole ` + + `premise is that it has never been run`, + `the model-backed check records ${verdicts.length} verdict row(s) and none of them is a PASS`, + ); +} + +// --------------------------------------------------------------------------- +// Claim 2 — the artifact does not outlive the run: the repository it is +// committed into is created under the OS temp directory. +// --------------------------------------------------------------------------- +{ + const scratch = body("lab/scratch-repo.mts"); + claim( + /\btmpdir\(\)/.test(scratch) && /mkdtempSync/.test(scratch), + `the scratch repository is no longer created under the OS temp directory, so the spec's ` + + `"the artifact does not outlive the run" row is stale`, + `the artifact's repository is created with mkdtempSync under tmpdir(), so it does not ` + + `outlive the machine's temp sweep`, + ); + claim( + !/git\(\s*"push"/.test(scratch) && !/\bgh\b/.test(scratch), + `the scratch repository now pushes or shells to gh, so the artifact may already leave the run`, + `nothing in the artifact's repository helper pushes or calls gh`, + ); +} + +// --------------------------------------------------------------------------- +// Claim 3 — the model-backed check runs on stores that do not survive it. +// --------------------------------------------------------------------------- +{ + const honesty = body("it-commits-from-the-seats-own-file/run.mts"); + claim( + /inMemoryStores\(\)/.test(honesty), + `the model-backed check no longer runs on inMemoryStores(), so ER-3's "durable across the ` + + `run" may already be met`, + `the model-backed check runs on inMemoryStores(), which ER-3 calls not durable`, + ); +} + +// --------------------------------------------------------------------------- +// Claim 4 — the declared channel is walked but never opened, while the two +// sibling labs that prove the path both open theirs. +// +// The positive half is what makes this a gap rather than a missing primitive: +// a claim that only said "devforce does not open a channel" is equally true of +// a framework with no channels in it. +// --------------------------------------------------------------------------- +{ + const opensHere = tree.filter(([, t]) => /\bopenChannels\s*\(/.test(t)).map(([p]) => p); + claim( + opensHere.length === 0, + `the devforce lab already opens a channel (${opensHere.join(", ")}), so the spec's channel ` + + `gap is stale`, + `no file in the devforce lab calls openChannels`, + ); + + const declared = tree.filter(([p]) => p.endsWith("/CHANNEL.md")); + claim( + declared.length >= 1, + `the devforce lab declares no CHANNEL.md, so there is no channel to call unused`, + `the devforce lab declares ${declared.length} channel file(s) that nothing opens`, + ); + + const siblingsThatOpen = SIBLINGS.filter((dir) => + filesUnder(join(REPO, dir)).some(([, t]) => /\bopenChannels\s*\(/.test(t)), + ); + claim( + siblingsThatOpen.length === SIBLINGS.length, + `only ${siblingsThatOpen.length} of ${SIBLINGS.length} sibling labs open a channel, so ` + + `"the path is already proven next door" is weaker than the spec claims`, + `both sibling labs (${SIBLINGS.join(", ")}) open a channel on the same primitive`, + ); +} + +// --------------------------------------------------------------------------- +// Claim 5 — nothing in the lab reads the browsable child-session surface +// FIX-1440 removes, so an unlanded FIX-1440 does not fence this work. +// +// `parentage` is provenance, which FIX-1440's own owner amendment keeps. +// --------------------------------------------------------------------------- +{ + const browse = tree.filter(([, t]) => /\/children\b|childSessions|listChildren/.test(t)); + claim( + browse.length === 0, + `the lab reads the child-session browse surface FIX-1440 removes (${browse + .map(([p]) => p) + .join(", ")}), so this work IS fenced by FIX-1440`, + `nothing in the lab reads the browsable child-session surface FIX-1440 removes`, + ); + + const provenance = tree.filter(([, t]) => /parentage:\s*\{\s*parentOf/.test(t)); + claim( + provenance.length >= 1, + `the lab no longer reads dispatch parentage at all, so the FIX-1440 finding needs re-deriving`, + `the lab reads dispatch parentage as provenance only (${provenance.length} site), which ` + + `FIX-1440's owner amendment explicitly keeps`, + ); +} + +function report() { + for (const line of evidence) console.log(` ok ${line}`); + for (const line of failures) console.log(` FAIL ${line}`); + console.log( + `\n${failures.length === 0 ? "PASS" : "FAIL"} — ${evidence.length} claim(s) held, ` + + `${failures.length} failed${plant === undefined ? "" : ` (plant: ${plant})`}`, + ); +} + +report(); +process.exit(failures.length === 0 ? 0 : 1); From 25e750753abd3ff7e8b6257247e591b548df2c40 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 22 Sep 2026 01:45:30 +0000 Subject: [PATCH 2/3] =?UTF-8?q?spec(FIX-1496):=20fold=20review=20round=201?= =?UTF-8?q?=20=E2=80=94=20three=20defects=20fixed,=20scope=20trimmed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Direction unchanged. Scope trimmed and three real defects fixed. Codex P1 — the acceptance condition was satisfiable by vacuous work. "A passing test exists" is met by any unrelated passing test, or by a vacuous test beside a broken greet, which left the ignores-the-brief control unable to fail — the control that makes D1 more than taste. The brief now names a contract (greet(name) from a named path, stated behaviour for empty and non-empty), and the acceptance check is the requester's: it imports the named export, asserts both behaviours, runs from outside the checkout, and the run cannot edit it. V5 grows to three controls that must all go red. Codex P1 — BR-8 conflated two dispatch kinds. Verified in code: the notify block runs once per declared member, but declares its own targets, and a member absent from the address map is recorded and skipped. So the rule was ambiguous rather than unsatisfiable. It now forbids only a board/harness WORK dispatch to the reviewer; a channel notification is expected. V2 grades the board dispatch record by flowId. Codex P2 — gap-check had a hole under a conclusion already passed upward. The pattern spelled childSessions lower-case and could not match listChildSessions, the canonical client method and the exact call claim 5 was about. Reproduced. Pattern is now case-insensitive and anchored on the canonical name; the case is a planted control. Re-run: the FIX-1440 conclusion still holds, but its evidence was not sound until now, and DECISIONS.md says so. Scope, per coordinator: channel driving moves from a sign-off decision to a recorded constraint (ER-DevForce's ratified wording carries it, verified); two artifact legs collapse to one automated gate with the PR as a documented human release run; running the never-run commit sibling splits out to FIX-1501. Also: gap-check gains an explicit sunset rule (deleted at S8, two claims move into the new goal), four more negative controls, one canonical figure with the rest cross-linked, and Cursor's three implementation notes recorded verbatim. Corrected two stale premises relayed in review rather than repeating them: ER-Collab's producer exists (FIX-1497; the merged epic spec is stale), and openChannels takes no orgId and needs no wrapper — FIX-1443's title says otherwise and is wrong about its own resolution; FIX-1442 is what closed it. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016qaz1VpRK4wCQzxpjzZ4Hz --- specs/issues/FIX-1496/BUSINESS-RULES.md | 26 +-- specs/issues/FIX-1496/DECISIONS.md | 85 ++++++--- specs/issues/FIX-1496/DOCS.md | 19 +- specs/issues/FIX-1496/PLAN.md | 137 +++++++++++---- specs/issues/FIX-1496/SPEC.md | 60 +++---- .../issues/FIX-1496/figures/the-four-gaps.svg | 13 +- specs/issues/FIX-1496/poc/gap-check/README.md | 40 ++++- specs/issues/FIX-1496/poc/gap-check/check.mjs | 164 ++++++++---------- 8 files changed, 338 insertions(+), 206 deletions(-) diff --git a/specs/issues/FIX-1496/BUSINESS-RULES.md b/specs/issues/FIX-1496/BUSINESS-RULES.md index f39dc30140..dae516973d 100644 --- a/specs/issues/FIX-1496/BUSINESS-RULES.md +++ b/specs/issues/FIX-1496/BUSINESS-RULES.md @@ -12,14 +12,15 @@ work. |---|---|---|---| | BR-1 | The run finishes and the goal's process exits | The artifact resolves at its address with nothing of the run still alive — no held checkout, no live store | Goal, on the real path | | BR-2 | The artifact is inspected for the lab's fingerprints | No graded token of its content appears in the lab's own code, its fixtures, or the prompt the seat was handed | Goal · the held-out discipline, inverted onto the product | -| BR-3 | The acceptance condition the brief stated is executed in the produced tree | It passes, **and** the same condition run against the base ref does not. A condition that was already true is no evidence | Goal · both halves, or the leg is vacuous | -| BR-4 | A run produces an artifact that does not satisfy the condition | The row does not settle `completed`. The artifact existing is never sufficient | Goal · control `ignores-the-brief` | +| BR-3 | The acceptance check the brief named is executed against the produced tree | It **imports the named export from the named path and asserts the named behaviours**, and passes — **and** the same check against the base ref fails. A condition already true before the run is no evidence | Goal · both halves, or the leg is vacuous | +| BR-3a | A run adds a passing test of its own, or a vacuous test beside a broken `greet` | Neither satisfies BR-3. The acceptance check is the requester's, is run from outside the checkout, and the run cannot edit or substitute for it | Goal · controls `unrelated-passing-test` and `vacuous-test` | +| BR-4 | A run produces an artifact that does not satisfy the acceptance check | The row does not settle `completed`. The artifact existing is never sufficient | Goal · control `ignores-the-brief` | | BR-5 | A run reports it stopped at its budget, having committed something | The row does not settle `completed`, exactly as today | Existing gate · control `stopped-at-limit` | -![Two lanes over the DevForce path showing which stages are unchanged and which four gaps close](figures/the-four-gaps.svg) - -The right-hand end of the *after* lane is this group: the artifact stage and the grading beside -it. BR-1 to BR-3 are the three properties [D1](DECISIONS.md#d1) names, one rule each. +This group is the right-hand end of the *after* lane in +[the four-gaps figure](SPEC.md#what-changes) — the artifact stage and the grading beside it. +BR-1 to BR-3 are the three properties [D1](DECISIONS.md#d1) names, one rule each. The figure +lives in `SPEC.md` and is not redrawn here. ## The channel is driven, not decorated @@ -27,7 +28,7 @@ it. BR-1 to BR-3 are the three properties [D1](DECISIONS.md#d1) names, one rule |---|---|---|---| | BR-6 | An operator posts on the feature channel | The channel opens, the post lands in its transcript, and the EM seat is reached | Goal | | BR-7 | The EM seat is reached by that post | It files exactly one row on the feature board, addressed to the `coder` assignee | Goal · one row, not one per member | -| BR-8 | The post reaches the other declared members | No board row is filed by any of them, and the reviewer seat is dispatched nothing | Goal · control `reviewer-files` | +| BR-8 | The post reaches the other declared members | No board row is filed by any of them, and **no board or harness work dispatch reaches the reviewer seat**. A channel *notification* delivery to a declared member is expected and is not what this forbids — the two are different dispatch kinds and only the work one is fenced | Goal · control `reviewer-files`, graded on the **board** dispatch record by `flowId` | | BR-9 | Nobody posts | No row is filed and no run happens. The board does not start itself | Goal · the negative half of BR-6 | | BR-10 | The channel is opened without an org | Refused at the transport door, as today | Existing gate (BR-17 there), unchanged | @@ -38,23 +39,24 @@ it. BR-1 to BR-3 are the three properties [D1](DECISIONS.md#d1) names, one rule | BR-11 | The proof runs | It runs on stores that are on disk, and the row, the run record and the transcript are all readable from a fresh process over the same file | Goal · the store is closed and re-opened | | BR-12 | The coder seat's run is dispatched | Its prompt carries every held-out token from the seat's own instructions, brief and skills — unchanged from today | Existing gate, and re-asserted here | | BR-13 | The harness is unavailable or unauthenticated | The proof **fails loudly**. It never degrades to a scripted stub | Goal · no stub fallback, deliberately | -| BR-14 | The pull-request leg is not configured | The proof runs its default leg and says so in its own output. It does not silently report the credentialed claim | Goal · the two legs are named in the verdict, never conflated | -| BR-15 | The same proof is re-run a year later | The default leg needs `git`, a temporary directory and a signed-in harness. It needs no `gh`, no token and no network | Read of the check's own stated requirements | +| BR-14 | The proof runs | It names which leg it ran. **CI runs only the temp-repository leg**; the credentialed pull-request leg is the **human release run** and is never inferred from whether `gh` happens to be installed | Goal · the verdict names the leg, and the two claims are never conflated | +| BR-15 | The same proof is re-run a year later | The automated leg needs `git`, a temporary directory and a signed-in harness. It needs no `gh`, no token and no network | Read of the check's own stated requirements | ## What the proof must not quietly become | # | When | Then | Proved by | |---|---|---|---| | BR-16 | The change is reviewed against the epic's fences | No L1 type, no `TaskStatus` value, no second work plane, no new substrate, nothing under `packages/` | Diff gate: every changed path is inside `goals/devforce-lab/` or `specs/issues/FIX-1496/` | -| BR-17 | The existing two checks are re-run after the change | Both still pass with their existing claims intact, and their verdict logs are appended to, never rewritten | CI · both siblings run | +| BR-17 | The existing two checks are looked at after the change | The model-free gate still passes with its claim intact. The model-backed sibling still type-checks and keeps its claim; **running it is [FIX-1501](https://linear.app/fixpoint-labs/issue/FIX-1501), not this issue**. Neither verdict log is rewritten | CI for the gate · type-check for the sibling | ## Failure taxonomy **Fatal, and loud:** an unavailable harness (BR-13), a tree that does not load, a base ref the scratch repository lacks — all refused before an attempt is charged. **Non-fatal and retried:** a run that finishes and leaves nothing re-pends the row with its reason and errors once the budget -is spent, exactly as the existing gate proves. **Never silent:** an unconfigured pull-request leg -(BR-14) degrades the *claim*, not the run, and the verdict says which leg it ran. +is spent, exactly as the existing gate proves. **Never silent:** running the automated leg rather +than the credentialed one (BR-14) degrades the *claim*, not the run, and the verdict says which +leg it ran. ## Acceptance criteria this issue owns diff --git a/specs/issues/FIX-1496/DECISIONS.md b/specs/issues/FIX-1496/DECISIONS.md index 7157e56cd3..474e130136 100644 --- a/specs/issues/FIX-1496/DECISIONS.md +++ b/specs/issues/FIX-1496/DECISIONS.md @@ -2,7 +2,7 @@ [Spec](SPEC.md) · **Decisions** · [Rules](BUSINESS-RULES.md) · [Plan](PLAN.md) · [Docs](DOCS.md) -What was considered, what was chosen, why, and what each locks in. Three decisions are the +What was considered, what was chosen, why, and what each locks in. **Two** decisions are the sign-off surface; everything else here is context for them. ## The tree @@ -14,11 +14,13 @@ flowchart TD D1 -.->|"rejected"| X2["a pull request on this repository
strongest evidence, but an agent opening PRs on production"] I --> D2["D2 · grade the product against a
pre-stated, executable condition"] D2 -.->|"rejected"| X3["keep grading only the prompt
proves the plumbing, never the work"] - I --> D3["D3 · the row is filed in answer to a
post on the feature channel"] - D3 -.->|"rejected"| X4["leave the channel declared and unused
ER-DevForce's own words go unmet"] + I --> R["driving the channel · not a decision
the epic already ratified it"] + R -.->|"not ours to reopen"| X4["defer channels to ER-Collab
would ship a proof the exit criterion does not accept"] ``` -Solid edges are what you are signing. Dashed edges lost, and the label says why. +Solid edges are what you are signing. Dashed edges lost, and the label says why. The third node +is **not** a fork: driving the channel is the epic's wording, recorded under +[Decided, not asked](#decided-not-asked). ## D1 · The artifact is a pull request on a separate throwaway repository, and "real" is defined by three falsifiable properties rather than by taste @@ -49,28 +51,52 @@ needs no credential. Say so and the pull-request leg becomes optional rather tha |---|---| | **Instead of** | Keeping the existing rule that only the prompt is graded and the product never is | | **Because** | The existing rule is right about what it guards — a model writes a plausible file without reading anything, so the file is no evidence a document was *read*. But ER-DevForce asks for a **work product**, and nothing that refuses to look at the product can prove one. The way out is not to grade the file's contents by eye; it is to grade it against a condition the *requester* wrote down before the run and a machine can execute | -| **Locks in** | The feature brief stops being flavour text and becomes a contract. Every future DevForce brief owes an executable done-condition, and a Lab task that cannot state one cannot be proved this way — which is a real limit on what this proof shape can cover | +| **Locks in** | The feature brief stops being flavour text and becomes a contract. Every future DevForce brief owes an executable done-condition **naming the behaviour it wants**, and a Lab task that cannot state one cannot be proved this way — which is a real limit on what this proof shape can cover | Both gradings stay, because they answer different questions. *Did the seat's own files reach the run?* is the prompt's job and is unchanged. *Did the run produce work someone asked for?* is the -condition's job and is new. A run that improvised without reading the brief fails the second -while possibly passing neither. +condition's job and is new. - -## D3 · The row is filed in answer to a post on the feature channel, not by calling the EM seat's action directly +**The condition has to be specific, independent and unreachable, or it proves nothing.** Three +properties, each learned from a way the weak version fails: -| | | -|---|---| -| **Instead of** | Leaving the declared channel walked-but-unopened, as it is today, and filing through the EM seat's own action | -| **Because** | ER-DevForce says *seats **and channels** used honestly*. Today the channel is a Markdown file the loader reads and nothing opens — honest by the letter, empty in fact. The post-reaches-a-seat path is already proven in two sibling labs on the same primitive, so closing this costs composition, not invention | -| **Locks in** | The proof now depends on the channel fan-out as well as the board hand-off, so a regression in either fails it. That is the point: it is one path end to end, and a proof that skipped the front door would have been reporting on a shorter path than the one we ship | +- **Specific.** *"The repository's tests pass"* is satisfied by any unrelated passing test, and + by a vacuous test sitting beside a broken `greet`. The condition names the module path, the + function, and the behaviour for both a non-empty and an empty name. +- **Independent.** It is the requester's check, not the run's. A run that writes its own test and + passes it has graded its own homework. +- **Unreachable from the run.** It is applied from outside the checkout, after the run, so the + agent cannot edit, delete or satisfy it by rewriting it. -**What would change my mind:** if [ER-Collab](../../epics/FIX-1457/BUSINESS-RULES.md#er-collab)'s -producer is filed this cycle and takes the channel leg explicitly, this becomes duplicated -evidence and should be dropped here to keep the path thinnest. Nothing has been built against it. +Without all three, `ignores-the-brief` ([BR-4](BUSINESS-RULES.md), V5) — the control whose whole +job is proving irrelevant work gets rejected — **cannot fail**, and a control that cannot fail is +decoration. The base-ref half still holds and still matters: the same check run against the base +ref fails because `src/greeting.js` does not exist there. ## Decided, not asked +- **The row is filed in answer to a post on the feature channel, not by + calling the EM seat's action directly — and this is not a fork.** + [ER-DevForce](../../epics/FIX-1457/BUSINESS-RULES.md#er-devforce) is worded *"One DevForce path + completes and produces a real artifact … **with seats and channels used honestly rather than + stubbed past**,"* and the epic's [people table and set table](../../epics/FIX-1457/SPEC.md) + repeat the phrase twice more. Channels are inside the ratified exit criterion, not an + embellishment on it, so deferring them would ship a proof that does not prove what the + criterion says. Today the channel is a Markdown file the loader walks and nothing opens — + honest by the letter, empty in fact — and the post-reaches-a-seat path is already proven in two + sibling labs on the same primitive, so closing it costs composition, not invention. **Not ours + to reopen locally** ([ER-17](../../epics/FIX-1457/BUSINESS-RULES.md#er-17)). + **Sequencing, not a fork:** ER-Collab's producer **exists** — + [FIX-1497](https://linear.app/fixpoint-labs/issue/FIX-1497), Backlog, not started. If it lands + the channel leg first, this proof **reuses** it rather than re-building it. *(The merged epic + spec still records ER-Collab and ER-DevForce as having **no producer**; both lines went stale + when FIX-1497 and this issue were filed. The epic amendment is the coordinator's, on its own PR + from `main` — do not read the merged row as current, here or anywhere.)* +- **One automated gate, not two.** The temp-repository leg is the **only** leg CI runs; the + credentialed pull-request leg is documented and named in the verdict as the **human release + run**. That keeps [BR-14](BUSINESS-RULES.md) and [BR-15](BUSINESS-RULES.md) honest — the claim + a run makes is still tied to the leg it ran — without carrying two lab code paths forever. + - **This is a delta on `goals/devforce-lab/`, not a new Lab tree.** Roughly four fifths of the path is built and passing; the issue's own fence says compose ([ER-11](../../epics/FIX-1457/BUSINESS-RULES.md), [ER-25](../../epics/FIX-1457/BUSINESS-RULES.md)). @@ -82,11 +108,12 @@ evidence and should be dropped here to keep the path thinnest. Nothing has been interim tax is already labelled as one; paying it for one more proof is cheaper than the risk. - **No package under `packages/` changes.** Everything the proof needs — the channel primitives, the harness manager, the `gh` completion probe — already ships. -- **The existing never-run commit check gets run too.** It is the cheaper half of the same - wiring, and leaving a written-but-unrun check beside a newly passing one is how the next reader - learns that unrun checks are normal here. - **Stores go on disk for the new proof only.** ER-3 asks for durable across the run; the model-free gate is a contract check and has no such requirement. +- **Running the never-run commit sibling is not this issue.** It is good hygiene and it is over + the *thinnest path, then stop* fence, so it is filed separately + ([FIX-1501](https://linear.app/fixpoint-labs/issue/FIX-1501), related, not blocking) rather + than carried here. ## Considered and dropped @@ -106,9 +133,16 @@ evidence and should be dropped here to keep the path thinnest. Nothing has been reads the browsable child-session surface FIX-1440 removes, and the one parentage read is the provenance form the owner amendment explicitly retains. FIX-1440 is `Spec Approved`, not shipped; that remains true and remains irrelevant here. -- **The gap table is four rows, not more** — [`poc/gap-check/`](poc/gap-check/README.md) asserts - all twelve facts the spec rests on, including a totality assertion over every TypeScript file - in the lab, and goes red under four planted defects. Run 2026-09-22: `PASS — 12 claims held`. + **The conclusion is unchanged; the evidence behind it was not sound until review round 1.** The + original pattern spelled `childSessions` with a lower-case c and so could not match + `listChildSessions`, the canonical client method and the exact call the claim was about — the + check could not have caught the case it was asked to check. Found by reproduction. The pattern + now covers the canonical name, the case is a planted control, and the re-run is green. Anyone + who took this as verified before 2026-09-22 was taking it on a check that could not fail. +- **The gap table is four rows, not more** — re-derivable by running + [`poc/gap-check/`](poc/gap-check/README.md), which asserts all twelve facts the spec rests on, + including a totality assertion over every TypeScript file in the lab. Dated runs and the + planted red states live in its README, not here. ## How it got here @@ -117,3 +151,8 @@ evidence and should be dropped here to keep the path thinnest. Nothing has been is a four-gap delta on the existing lab rather than a new DevForce path, and the artifact question the Architect left open is answered as a pull request with three falsifiable properties. +- **Review round 1** — scope trimmed, direction unchanged. The channel leg stopped being a + sign-off decision and became a recorded constraint, because the epic's ratified ER-DevForce + wording already carries it. The two artifact legs collapsed to **one automated gate** plus a + documented human release run. Running the neighbouring commit check left the plan for + [FIX-1501](https://linear.app/fixpoint-labs/issue/FIX-1501). diff --git a/specs/issues/FIX-1496/DOCS.md b/specs/issues/FIX-1496/DOCS.md index 9e77bc0125..18dca3fd20 100644 --- a/specs/issues/FIX-1496/DOCS.md +++ b/specs/issues/FIX-1496/DOCS.md @@ -35,16 +35,19 @@ Add two rows to the existing table and one line of framing above it. Draft: | File | Why it is the lab's | |---|---| -| the channel door in `host.mts` | The org is not threaded through `openChannels` yet ([FIX-1412](https://linear.app/fixpoint-labs/issue/FIX-1412)), so the session client is wrapped to inject one. The same wrap the two sibling labs carry, for the same reason | -| the two artifact legs in `scratch-repo.mts` | Whether the artifact leaves the machine is an operator's choice, not the lab's. The default leg stays a temporary repository so the check is re-runnable with no credential; the credentialed leg gives the artifact an address a person can open | +| the channel door in `host.mts` | Which channel this lab opens, and when, is the app's. `openChannels` takes no `orgId` and needs no wrapper — the server-created session carries the organization from the verified principal ([FIX-1442](https://linear.app/fixpoint-labs/issue/FIX-1442)) | +| the bare clone in `scratch-repo.mts` | Where an artifact has to survive to is the lab's question, not the framework's. A temp-directory repository plus a bare clone the run pushes to keeps the check re-runnable with no credential while still giving the artifact an address that outlives the process | ### UPDATE · `goals/devforce-lab/lab/README.md` · *What it works around* Append one paragraph. Draft: -> **The pull-request leg needs a credential, and the check says which leg it ran.** A proof that -> quietly ran the weaker leg and reported the stronger one would be worse than no proof, so the -> leg is named in the verdict rather than inferred from whether `gh` happened to be installed. +> **One automated leg, and the verdict says which leg ran.** CI runs the local leg: the run +> pushes to a bare clone, and the artifact resolves there after the process exits, with no +> network and no credential. The pull-request release run is the same path with a real remote, +> and it is a human release step rather than a second leg the lab carries. A proof that quietly +> ran the weaker leg and reported the stronger one would be worse than no proof, so the leg is +> named in the verdict rather than inferred from whether `gh` happened to be installed. ### CREATE · `goals/devforce-lab/it-ships-an-artifact-a-person-can-open/goal.md` @@ -57,9 +60,9 @@ this spec and are drafted here. > Somebody posts one line into a channel a Markdown file declared. A coordinator seat reads the > post and files one row. A working seat the row names wakes in a checkout of its own, runs a -> real coding agent from a prompt built out of its own files, and opens a pull request — at an -> address that still resolves when the run is over, carrying work that satisfies a condition the -> brief stated before the run started. +> real coding agent from a prompt built out of its own files, and leaves its work at an address +> that still resolves when the run is over — work that satisfies an acceptance check the brief +> named before the run started, written by the requester and applied from outside the checkout. **What this establishes, and what it does not** (draft): diff --git a/specs/issues/FIX-1496/PLAN.md b/specs/issues/FIX-1496/PLAN.md index c979779414..94dc91fff2 100644 --- a/specs/issues/FIX-1496/PLAN.md +++ b/specs/issues/FIX-1496/PLAN.md @@ -9,14 +9,15 @@ Written for the implementing agent. IDs cross-reference [BUSINESS-RULES.md](BUSI | ID | Package · role | Change | Rules | |---|---|---|---| -| S1 | `goals/devforce-lab/lab` · the authored brief | Rewrite the feature brief around an **executable** done-condition: an exported greeting function plus a test the repository's own runner picks up, and the existing held-out marker carried into the pull request body. Keep the marker — the prompt is still graded for it (D2) | BR-3 BR-12 | -| S2 | `goals/devforce-lab/lab` · the host | Open the declared feature channel (`channelInstances` + `openChannels`, as both sibling labs do) and expose posting on it. The EM seat files its row in answer to the post rather than through a direct action call (D3) | BR-6 BR-7 BR-8 BR-9 BR-10 | -| S3 | `goals/devforce-lab/lab` · the artifact's repository | Give the scratch repository a **real** remote and a second construction mode: today's temp-dir repository for the default leg, and a configured remote for the credentialed leg. Name the leg in what the helper returns so the verdict can report it (D1) | BR-1 BR-14 BR-15 | -| S4 | `goals/devforce-lab/lab` · the phase | Make the done-condition a slot rather than a fixed commit probe. Default stays *a commit the base ref lacks*; the credentialed leg uses the `gh` completion probe `labs/conductor` already ships. **Do not re-implement it** | BR-1 BR-4 BR-5 | -| S5 | `goals/devforce-lab/lab` · the acceptance gate | Execute the brief's condition in the produced tree, and the same condition against the base ref. Both halves, or the leg proves nothing (D2) | BR-3 BR-4 | +| S1 | `goals/devforce-lab/lab` · the authored brief | Rewrite the feature brief around a **named contract**: export `greet(name)` from a named path, with stated behaviour for a non-empty and an empty name. Keep the held-out marker — the prompt is still graded for it (D2, BR-12) | BR-3 BR-12 | +| S1a | `goals/devforce-lab/lab` · the acceptance check | **New, and it is the requester's, not the run's.** A small check that imports the named export from the named path and asserts both behaviours. It lives with the lab, **outside the run's checkout**, and is executed against the produced tree after the run — so the agent cannot edit it, delete it, or substitute a test of its own for it | BR-3 BR-3a BR-4 | +| S2 | `goals/devforce-lab/lab` · the host | Open the declared feature channel (`channelInstances` + `openChannels`, as `manager-queue-lab` does) and expose posting on it. The EM seat files its row in answer to the post rather than through a direct action call. **No org wrapper** — see *At implement time* | BR-6 BR-7 BR-8 BR-9 BR-10 | +| S3 | `goals/devforce-lab/lab` · the artifact's repository | **One automated path.** The temp-directory repository stays, and gains a **bare clone at a declared path** that the run pushes to — so the artifact resolves after the process exits without needing a network or a credential. The helper names the leg it built so the verdict can report it (D1, BR-14) | BR-1 BR-14 BR-15 | +| S4 | `goals/devforce-lab/lab` · the phase | Done-condition stays the existing commit probe — *a commit the base ref lacks*, now read from the pushed bare clone. **No `gh` probe and no second phase path in the lab.** The credentialed pull-request release run is documented in `goal.md`, using conductor's existing `prExists` slot, and is not wired as a CI leg | BR-1 BR-4 BR-5 | +| S5 | `goals/devforce-lab/lab` · the acceptance gate | Execute S1a's check against the produced tree, and against the base ref. Both halves, or the leg proves nothing (D2). The two executions share one checkout setup | BR-3 BR-3a BR-4 | | S6 | `goals/devforce-lab/it-ships-an-artifact-a-person-can-open/` · **new sibling goal** | The runner, its `goal.md` (outcome · input · signal · anti-game · controls · verdict log), and the controls below. Reuses `openLab` with the S2–S5 options; **does not edit either existing check** | all | -| S7 | `goals/devforce-lab/lab/README.md` | Extend *What this directory owns* and *What it works around* for the channel door and the two legs. Record that the artifact now leaves the run | — | -| S8 | The verdict logs | Run the new proof **and** the never-run `it-commits-from-the-seats-own-file`, and append a dated row to each. Appending only — never rewrite an existing row | BR-17 | +| S7 | `goals/devforce-lab/lab/README.md` | Extend *What this directory owns* and *What it works around* for the channel door and the artifact's bare clone. Record that the artifact now outlives the run | — | +| S8 | The verdict log · and `gap-check`'s sunset | Run the new proof and append **one** dated row to its own `goal.md`. Appending only. **Then delete `specs/issues/FIX-1496/poc/gap-check/`** — see the sunset rule below. Running the never-run commit sibling is **not here**: [FIX-1501](https://linear.app/fixpoint-labs/issue/FIX-1501) | BR-17 | **Nothing is removed.** Named explicitly because tenet 3 expects the question asked: the two existing checks keep their claims, the board stays declared in code, and the `coder` kind's @@ -27,37 +28,61 @@ second board declaration stays the labelled interim tax it already is ```mermaid flowchart TD - S1["S1 · brief gains an executable condition"] --> S5["S5 · the acceptance gate"] + S1["S1 · brief names a contract"] --> S1A["S1a · the requester's acceptance check"] + S1A --> S5["S5 · the acceptance gate, both halves"] S2["S2 · open and drive the channel"] --> S6["S6 · the new sibling goal"] - S3["S3 · the artifact's repository, two legs"] --> S4["S4 · done-condition as a slot"] + S3["S3 · the artifact outlives the run"] --> S4["S4 · done-condition, unchanged"] S4 --> S6 S5 --> S6 - S6 --> S8["S8 · run both, record both verdicts"] + S6 --> S8["S8 · run it, record the verdict, delete gap-check"] S6 --> S7["S7 · the lab README"] ``` -One PR. S1–S5 are independent of each other and all feed S6; S8 is the proof itself and is last -because it is the only step that needs a credentialed machine. +One PR. S1→S1a→S5 is the only chain; S2 and S3→S4 are independent of it and of each other, and +all three feed S6. S8 is the proof itself and is last because it is the only step that needs a +**signed-in coding harness** — it needs no credential and no network beyond that. ## Checks | ID | Runs after | Passes when | |---|---|---| | V1 | S2 | BR-6, BR-7, BR-9. A post opens the channel, lands in the transcript, and produces **exactly one** row. No post, no row | -| V2 | S2 | BR-8. Control `reviewer-files`: the reviewer seat is made to answer the post by filing; the check goes red on the dispatch record, not merely on the transcript | -| V3 | S3, S4 | BR-14, BR-15. The default leg runs with no `gh`, no token and no network, and the verdict names which leg ran. Both legs exercised | -| V4 | S5 | BR-3, **both halves**: the condition passes in the produced tree and fails against the base ref. Control `already-passing`: a condition true before the run goes red | -| V5 | S5 | BR-4. Control `ignores-the-brief`: a run that commits something unrelated does not settle the row `completed` | +| V2 | S2 | BR-8. Control `reviewer-files`: the reviewer seat is made to answer the post by filing; the check goes red on the **board** dispatch record (by `flowId`), not merely on the transcript. **Grade the board dispatch, never the notify delivery** — a channel notification to a declared member is expected, so a check that counted any dispatch to the reviewer would go red on correct behaviour | +| V3 | S3, S4 | BR-14, BR-15. The automated leg runs with no `gh`, no token and no network; the artifact resolves from the bare clone after the run's process has exited; and the verdict names the leg. **One leg, not two** | +| V4 | S5 | BR-3, **both halves**: the acceptance check passes against the produced tree and fails against the base ref. Control `already-passing`: a check true before the run goes red | +| V5 | S5 | BR-4, BR-3a. Three controls, because one is not enough here: `ignores-the-brief` (commits something unrelated), `unrelated-passing-test` (adds a passing test that never imports `greet`), and `vacuous-test` (a passing test beside a `greet` that returns the wrong string). **All three must go red**; if any passes, the acceptance check is still too weak to carry D2 | | V6 | S6 | BR-2. Every graded token of the artifact's content is absent from the lab's `.mts` files, its fixtures and the prompt — asserted **before** any verdict is read | | V7 | S6 | BR-11. The store is closed and a fresh process reads the same row, run record and transcript | | V8 | S6 | BR-13. Control `no-harness`: the check fails loudly and does not fall back to a stub | -| VG | S8 | **The goal.** One command on a credentialed machine: a post produces a pull request whose address resolves after the process exits, whose content the lab did not author, and which satisfies the brief's condition. `goals/devforce-lab/it-ships-an-artifact-a-person-can-open/run.mts` | +| VG | S8 | **The goal.** One command: a post produces an artifact whose address resolves after the process exits, whose content the lab did not author, and which satisfies the brief's condition. `goals/devforce-lab/it-ships-an-artifact-a-person-can-open/run.mts` | | V9 | S8 | BR-16. Diff gate: every changed path is under `goals/devforce-lab/` or `specs/issues/FIX-1496/`. Nothing under `packages/` | -| V10 | S8 | BR-17. Both existing siblings still pass, claims unchanged | +| V10 | S8 | BR-17. The model-free gate still passes with its claim intact; the model-backed sibling still type-checks. Running it is [FIX-1501](https://linear.app/fixpoint-labs/issue/FIX-1501) | **Every control must be seen red.** A green check nobody has watched fail is not evidence (tenet 7), and the existing labs record each control's red state in the verdict log — match that. +## `gap-check` sunsets at S8 — delete it, and carry two claims forward + +**The checker measures a snapshot, so it goes red on the very PR that closes the gaps it +measures.** That is by construction, not a defect: claims 2–4 assert *`inMemoryStores()` is in +use*, *nothing calls `openChannels`*, and *the artifact's repository is a temp directory*, and +S1–S5 make all three false. A check that fails when the work succeeds is worse than no check, so +it does not survive the work. + +**The rule: at S8, delete `specs/issues/FIX-1496/poc/gap-check/` entirely.** Not disabled, not +left failing, not rewritten in place — deleted, in the same PR that closes the last gap. It did +its job before review, which is when a hand-counted factual base is worth anything. + +**Two of its twelve claims have value beyond the snapshot, and move rather than die:** + +| Claim | Where it goes | +|---|---| +| The **totality** discipline — every tracked file classified as in-scope or deliberately out, so a file nobody listed cannot hide | The new goal's **leg 0**, where the sibling labs already put their held-out scan (`V6`) | +| The **FIX-1440 provenance** read — parentage is read as provenance, never as a work control plane | The new goal's **anti-game** section as a stated invariant, and [ER-13](../../epics/FIX-1457/BUSINESS-RULES.md) already fences it at epic altitude | + +The remaining ten are snapshot facts whose whole purpose was to make this spec checkable at +review time. They expire on merge, and saying so here is what stops someone reviving them. + ## Pinned names · the only two | Where | Name | Why pinned | @@ -71,9 +96,11 @@ Everything else is yours to name. | Rule | Because | |---|---| -| The artifact's content is graded **only** against the brief's condition, never for tokens | A model emits a token it was told to emit. The condition is the only grading that a run which never read the brief cannot fake (D2) | -| BR-3 runs both halves — the condition against the produced tree *and* against the base ref | A condition that was already true reports PASS on a run that did nothing. This is the vacuous-green shape the sibling labs each found the hard way | -| The credentialed leg and the default leg are named in the verdict, never merged into one claim | A proof that quietly ran the weaker leg and reported the stronger one is worse than no proof (BR-14) | +| The artifact's content is graded **only** against the acceptance check, never for tokens | A model emits a token it was told to emit. The check is the only grading that a run which never read the brief cannot fake (D2) | +| The acceptance check names the export, the path and the behaviour — never "some test passes" | *"The repository's tests pass"* is satisfied by an unrelated passing test and by a vacuous test beside a broken `greet`, which leaves `ignores-the-brief` unable to fail. A control that cannot fail is decoration (BR-3a) | +| The acceptance check lives outside the run's checkout and the run cannot edit it | A check the agent can reach is a check the agent can satisfy by rewriting. It is the requester's, applied after the run | +| BR-3 runs both halves — the check against the produced tree *and* against the base ref | A condition that was already true reports PASS on a run that did nothing. This is the vacuous-green shape the sibling labs each found the hard way | +| The verdict names the leg it ran, and CI carries **one** | A proof that quietly ran the weaker leg and reported the stronger one is worse than no proof (BR-14). Two lab code paths is the ongoing cost that buys nothing once the leg is named | | No stub fallback when the harness is missing | A model-backed check that degrades to a scripted run is the exact failure its model-free sibling exists to detect. The existing check already states this; keep it | | Every changed path stays inside `goals/devforce-lab/` or this spec | [ER-11](../../epics/FIX-1457/BUSINESS-RULES.md) and [ER-25](../../epics/FIX-1457/BUSINESS-RULES.md). A package change here is a substrate epic wearing a polish label, and V9 is the fence | | Compose `openChannels` and conductor's `gh` probe; do not re-implement either | [ER-4](../../epics/FIX-1457/BUSINESS-RULES.md). A second copy of a proven path is a second thing to keep true | @@ -86,14 +113,16 @@ goal's `goal.md`, both internal evidence. ## Sketch · pseudocode, illustrative, react to the shape +The path itself is [the four-gaps figure](SPEC.md#what-changes) and is not redrawn here. What the +sketch adds is the **wiring and the assertion order**, which the figure does not show: + ``` open the lab, as today, plus: stores ← on disk, under a path this check made (BR-11) - channels ← open the declared feature channel (D3) - repository ← default: a temp repo, as today - credentialed: a repo with a real remote (D1) - doneCondition ← default: a commit the base ref lacks, as today - credentialed: conductor's gh probe (D1) + channels ← open the declared feature channel (the epic's wording) + repository ← temp repo, as today, plus a bare clone at a + declared path that the run pushes to (D1, one leg) + doneCondition ← unchanged: a commit the base ref lacks (S4) the proof: post on the feature channel (BR-6) @@ -111,16 +140,39 @@ TypeScript file in the lab and four planted defects each watched going red. Run `node specs/issues/FIX-1496/poc/gap-check/check.mjs`. **The premise held**: all twelve green on 2026-09-22, including the one the brief flagged as worth checking — FIX-1440 does not fence this work. Nothing in the design changed because of it; it is here so a reviewer can re-run the -argument instead of re-reading it. +argument instead of re-reading it. **It is deleted at S8** — see the sunset rule above. ## At implement time -- **Confirm how a channel post reaches a seat's action.** The plan names the seam, not the - entry. `goals/pentest-lab/lab/host.mts` and `goals/manager-queue-lab/lab/host.mts` both do - this; copy whichever is closer. The fan-out reaches *every* declared member, so decide there - what the coder and reviewer seats do with a post — BR-8 only requires that neither files. -- **The org is not threaded through `openChannels`** (FIX-1412). Both sibling labs wrap the - session client to inject it. Wrap, do not work around, and do not fix FIX-1412 here. +- **The notify seam, as verified in code — read this before wiring S2.** + `defineChannelFlow` keeps the member walk and runs the `notify` block **once per declared + member per post** (`packages/workforce/src/channel/channel-flow.ts`). But the block **declares + its own dispatch targets**, because the dispatch seam refuses a target read out of stored data: + `goals/pentest-lab/lab/notify.mts` carries a static `addresses` map, and **a member absent from + it is recorded and skipped, not dispatched to**. So there are two distinct things a post can do + to the reviewer — invoke the notify block for it (always), and dispatch work to it (only if the + app's policy says so). **BR-8 forbids only the second.** You may leave the reviewer out of the + addresses map entirely, or include it and give it a true no-op; either satisfies BR-8, and V2 + grades the board dispatch record regardless. +- **`openChannels` takes no `orgId`, and needs none. Write no wrapper.** Call it exactly as + `goals/manager-queue-lab/lab/host.mts:467` does: + `await openChannels(channels, { client, userId })`. Read in code rather than inferred, because + two plausible-sounding versions of this are both wrong: + - *"Wrap the session client to inject the org"* is the **old** workaround. FIX-1412 is Done. + - *"Pass `orgId` to `openChannels`"* does not exist either — `OpenChannelsOptions` has no such + field, and `channel-binder.ts:681` says so outright: *"The binder no longer takes an `orgId` + at all, and never did have the authority to choose one."* **FIX-1443's title — + *"now that openChannels takes an orgId"* — is wrong about its own resolution**, and is the + trap that produces the second wrong version. What actually closed it was + [FIX-1442](https://linear.app/fixpoint-labs/issue/FIX-1442): the server-created session always + carries an organization from the verified principal, so a board always has an address. + + If the org does not in fact reach where the seats read org-scoped documents, **that is a real + finding — report it, do not reinstate the wrapper.** +- **Do not copy `goals/pentest-lab/lab/host.mts`'s channel setup.** Its `openChannels` call is + current, but it still carries `omitOrgWrap` and a header comment describing FIX-1412 as an open + gap. Some of that is a deliberate control and some is stale; either way it reads as current + practice and is not. `goals/manager-queue-lab/lab/host.mts` is the reference. - **Check whether [FIX-1440](https://linear.app/fixpoint-labs/issue/FIX-1440) has landed.** It was `Spec Approved` and unshipped when this was written, and [`poc/gap-check/`](poc/gap-check/README.md) claim 5 says it does not fence this work. Re-run @@ -130,12 +182,29 @@ argument instead of re-reading it. Devtool-visible. Soft; not a gate. - **`gh` is not installed on every runner.** It was absent from the machine this spec was written on. Conductor's probe already fails with an instruction rather than a stack trace; - make sure the default leg is what runs when it is missing. + make sure the automated leg needs none of it. + +## Notes from review + +Below-the-bar feedback from the spec PR, recorded verbatim for the implementer. **Inputs, not +instructions** — adopt, adapt or discard; you owe no justification for discarding one. + +- "Non-EM seats must be true no-ops in channel fan-out." — Cursor ([#2023](https://github.com/fixpoint-labs/flow-state-dev/pull/2023)) +- "BR-3/S5's two condition executions should share one checkout setup." — Cursor ([#2023](https://github.com/fixpoint-labs/flow-state-dev/pull/2023)) +- "V6's token scan reads the lab tree once." — Cursor ([#2023](https://github.com/fixpoint-labs/flow-state-dev/pull/2023)) + +A note that turns out to reveal a design problem is a spec blind spot — surface it and fold it +back, per the challenger discipline in `issue-implement`. ## Follow-ups - `labs/conductor`'s own `implement-phase-opens-a-pr` goal has **no verdict-log rows at all** — so "conductor proves the `gh` probe" is a claim about a code path, not a recorded run. Out of scope here; worth a ticket against the conductor lab. +- **`goals/pentest-lab/lab/host.mts` reads as stale on the org question** — `omitOrgWrap` and a + header comment still describe [FIX-1412](https://linear.app/fixpoint-labs/issue/FIX-1412) as an + open gap, though it and [FIX-1443](https://linear.app/fixpoint-labs/issue/FIX-1443) are both + Done. Not this issue's to clean, but the next reader of that file will be misled the same way + this spec nearly was. - The `coder` kind declaring the feature board a second time is an interim L1 tax carved onto [FIX-1408](https://linear.app/fixpoint-labs/issue/FIX-1408). Unchanged by this issue, still owed. diff --git a/specs/issues/FIX-1496/SPEC.md b/specs/issues/FIX-1496/SPEC.md index 4d31d2f6ad..399ae8e703 100644 --- a/specs/issues/FIX-1496/SPEC.md +++ b/specs/issues/FIX-1496/SPEC.md @@ -12,7 +12,7 @@ Feature · `goals/devforce-lab` only · **small** · 1 PR · epic [FIX-1457](../ | **asks what the workforce actually produced** | A commit in a temporary directory on whatever machine ran it, swept away with the rest of `/tmp` | A pull request with a title, a body and a diff, at an address that still resolves after the run is over | | **asks whether the work is any good, not merely that it exists** | Nothing grades the product. The check grades the *prompt*, deliberately, so a model that improvised a plausible file still passes | The brief names an acceptance condition **before** the run, and the check executes it against what the run produced. Improvising does not pass | | **asks whether the seats really used their channels** | The feature channel is a Markdown file the loader walks and nothing ever opens | A post on the feature channel is what starts the work. The channel is driven, not decorated | -| **has to re-run this proof a year from now** | Impossible: it has never been run once | One command. The credentialed pull-request leg is a flag; the default leg needs no network, no token and no `gh` | +| **has to re-run this proof a year from now** | Impossible: it has never been run once | One command, and it is what CI runs: no network, no token, no `gh`. The pull-request release run is the same path pointed at a real remote | **Why now.** W5's done condition is three exit proofs and none has been run ([FIX-1457](../../epics/FIX-1457/SPEC.md)). ER-DevForce is the one that decides whether @@ -26,7 +26,8 @@ and letting what it makes survive. The five stages are the same in both lanes, and the middle three are untouched. Only the two ends move, plus two properties of the run. **Nothing new is built** — the four gaps are the whole -issue ([D1](DECISIONS.md#d1), [D2](DECISIONS.md#d2), [D3](DECISIONS.md#d3)). +issue ([D1](DECISIONS.md#d1), [D2](DECISIONS.md#d2), and the channel leg the epic already +[settled](DECISIONS.md#decided-not-asked)). **The brief the coder seat works from, as its author writes it** — this is the surface that turns "a file exists" into "work a person would accept": @@ -37,13 +38,15 @@ issue ([D1](DECISIONS.md#d1), [D2](DECISIONS.md#d2), [D3](DECISIONS.md#d3)). Add a greeting module to the repository you are working in. - Write the file `GREETING.md`, and make its first line exactly: -+ Export a `greet(name)` function that returns a greeting, and cover it with a -+ test the repository's own test runner picks up. ++ Export a function `greet(name)` from `src/greeting.js`. For any non-empty ++ name it returns `Hello, !`; for an empty name it returns `Hello!`. + + ## Done when + -+ The repository's test run reports at least one passing test and no failing -+ ones. Before your change it reports none, so an untested module is not done. ++ The acceptance check accompanying this brief imports `greet` from that path ++ and asserts both behaviours, and passes. It is run from outside your ++ checkout and you cannot edit it. Adding a test of your own does not ++ substitute for it. + + Put this marker in the body of the pull request you open: @@ -51,24 +54,17 @@ issue ([D1](DECISIONS.md#d1), [D2](DECISIONS.md#d2), [D3](DECISIONS.md#d3)). ``` The marker stays because the existing check grades the *prompt* for it, which is how we know the -seat read its own files. What is new is everything above it: a condition a machine can run. - -## How the proof reaches an artifact - -```mermaid -flowchart LR - P["an operator posts on the feature channel"] --> E["the EM seat · files one row"] - E --> B["the feature board · one row, one assignee"] - B -->|"cross-flow hand-off"| C["the coder seat · its own checkout"] - C --> H["a real coding agent · prompt built from the seat's own files"] - H --> A["a pull request · pushed, opened, addressable"] - A --> V["the acceptance condition the brief stated, executed"] - R["the reviewer seat · declared, never reached"] -.->|"still never dispatched"| B -``` - -Everything from the EM seat to the coding agent exists and passes today under a scripted stub. -The two ends are what this issue adds, and the dashed edge is the negative claim the existing -gate already holds. +seat read its own files. What is new is everything above it: **a named contract and an +independent check the run cannot reach.** A brief that only said *"a passing test exists"* would +be satisfied by any unrelated passing test — which would leave the `ignores-the-brief` control +unable to fail, and that control is the whole reason [D1](DECISIONS.md#d1) is more than taste +([BR-3](BUSINESS-RULES.md), [BR-4](BUSINESS-RULES.md)). + +**The figure above is the one canonical picture of this path**, and the other documents point at +it rather than redrawing it. Everything from the EM seat to the coding agent exists and passes +today under a scripted stub; the two ends are what this issue adds. The reviewer seat stays +declared and never dispatched, which is a negative claim the existing gate already holds +([BR-8](BUSINESS-RULES.md)). ## What stays as it is @@ -94,10 +90,16 @@ gate already holds. 2. **[D2](DECISIONS.md#d2) · The proof grades what the run produced, against a condition the brief stated first and a machine executes.** *If wrong:* every future DevForce brief owes an executable acceptance condition, and a brief that cannot state one cannot be proved this way. -3. **[D3](DECISIONS.md#d3) · The row is filed in answer to a post on the feature channel, not by - calling the EM seat's action directly.** *If wrong:* one extra leg in a proof whose whole - virtue is thinness, for a claim ER-Collab may cover anyway. **Open: none.** Number 1 is the one to weigh — it is the question the Architect deliberately left -open, answered here as a proposal with a falsifier rather than left open again. The reasoning and -what lost: [DECISIONS.md](DECISIONS.md). The cases: [BUSINESS-RULES.md](BUSINESS-RULES.md). +open, answered here as a proposal with a falsifier rather than left open again. + +**Not asked, because the epic already answered it.** Driving the feature channel is inside +[ER-DevForce](../../epics/FIX-1457/BUSINESS-RULES.md#er-devforce)'s ratified wording — *"with +seats and channels used honestly rather than stubbed past"* — so it is a constraint this spec +obeys, not a call it makes ([Decided, not asked](DECISIONS.md#decided-not-asked)). Same section +records the other trim: **one automated gate**, with the credentialed pull-request leg documented +as the human release run rather than a second path CI carries. + +The reasoning and what lost: [DECISIONS.md](DECISIONS.md). The cases: +[BUSINESS-RULES.md](BUSINESS-RULES.md). diff --git a/specs/issues/FIX-1496/figures/the-four-gaps.svg b/specs/issues/FIX-1496/figures/the-four-gaps.svg index bf784dbbf0..bd400730e9 100644 --- a/specs/issues/FIX-1496/figures/the-four-gaps.svg +++ b/specs/issues/FIX-1496/figures/the-four-gaps.svg @@ -1,5 +1,5 @@ - + diff --git a/specs/issues/FIX-1496/poc/gap-check/README.md b/specs/issues/FIX-1496/poc/gap-check/README.md index 22c60e0214..e3c7390261 100644 --- a/specs/issues/FIX-1496/poc/gap-check/README.md +++ b/specs/issues/FIX-1496/poc/gap-check/README.md @@ -18,7 +18,7 @@ here instead: | The artifact's repository is created under the OS temp directory, and nothing pushes it | "The artifact does not outlive the run" | | The model-backed check runs on `inMemoryStores()` | ER-3's *durable across the run* is unmet | | Nothing in the lab calls `openChannels`, one `CHANNEL.md` is declared, and **both** sibling labs do call it | "The channel is walked but never driven, and the path is proven next door" | -| Nothing reads the browsable child-session surface; parentage is read as provenance only | FIX-1440 does not fence this work | +| Nothing reads the browsable child-session surface — pattern covers the canonical `listChildSessions` — and parentage is read as provenance only | FIX-1440 does not fence this work | ## Run it @@ -44,13 +44,45 @@ catch. `--plant list` prints them. Observed 2026-09-22 on `spec/FIX-1496`: | `channel-is-opened` | **FAIL** on the channel gap alone; the other 11 hold | | `durable-stores` | **FAIL** on the durability gap alone; the other 11 hold | | `model-check-passed` | **FAIL** on "never run" alone; the other 11 hold | +| `browse-surface` | **FAIL** on the FIX-1440 claim alone; the other 11 hold | +| `provenance-gone` | **FAIL** on the provenance read alone; the other 11 hold | +| `sibling-lost-channel` | **FAIL** on the "proven next door" half alone; the other 11 hold | +| `artifact-pushed` | **FAIL** on "nothing pushes" alone; the other 11 hold | Each control perturbs one thing and goes red on one claim, so a red here names its own cause. Plants are applied to the in-memory copy of the tree; nothing on disk is touched, so a control run cannot leave the repository perturbed. +## What these controls do and do not prove — read before trusting a green + +**A control that plants a comment tests the pattern, not the wiring.** `channel-is-opened` appends +a commented-out `openChannels(...)` line, so it proves the *detector* fires on that spelling. It +says nothing about whether opening a channel would actually work, and a reader should not take a +red here as integration coverage. The same is true of `browse-surface` and `artifact-pushed`. +Every claim in this script is a **textual** claim about the repository; none of them runs the lab. + +**`browse-surface` exists because this script already had exactly that hole.** The first version +of the FIX-1440 pattern spelled `childSessions` with a lower-case c, so it could not match +`listChildSessions` — the canonical client method +(`packages/client/src/session-client/sessions.ts:125`) and the precise call the claim was asked +about. All twelve claims reported green with such a call planted. It was found by someone +**reproducing** the case, not by reading the regex, and it survived a totality assertion and four +controls. The pattern is now case-insensitive and anchored on the canonical name, and the case is +a planted control so the green can be watched going red. + +**The lesson generalises and is worth stating:** the four original controls covered the claims +the author thought to perturb, which is exactly the set least likely to contain the hole. The four +added here close the gap for claims 4–5; the remaining unplanted assertions (the goal count, the +model-backed count, `tmpdir`, the `CHANNEL.md` count) are simple existence reads whose failure +mode is visible in the printed evidence line. + ## Limits -It reads the repository, not a running workforce. It can tell you the channel is never opened; -it cannot tell you the proof would pass once it is. That is the goal check's job, and building -it is what this issue is for. +It reads the repository, not a running workforce. It can tell you the channel is never opened; it +cannot tell you the proof would pass once it is. That is the goal check's job, and building it is +what this issue is for. + +**It is deleted at S8** (`PLAN.md` → the sunset rule). Its claims describe a snapshot that S1–S5 +make false by design, so it would go red on the very PR that closes the gaps it measures. Two +claims — the totality discipline and the FIX-1440 provenance read — move into the new goal rather +than dying with it. diff --git a/specs/issues/FIX-1496/poc/gap-check/check.mjs b/specs/issues/FIX-1496/poc/gap-check/check.mjs index d3e4e7c722..f81275401f 100644 --- a/specs/issues/FIX-1496/poc/gap-check/check.mjs +++ b/specs/issues/FIX-1496/poc/gap-check/check.mjs @@ -1,30 +1,11 @@ #!/usr/bin/env node /** - * Re-derive FIX-1496's factual base from the repository, rather than trusting - * the spec's prose. + * Re-derive FIX-1496's factual base from the repository. * - * The spec's whole argument is a gap table: ER-DevForce is mostly built, and - * exactly four things are missing. A hand-counted table like that does not - * converge by being argued about in review — each round corrects one row and - * leaves the ones nobody looked at. So it gets a checker, and the checker gets - * the two properties a hand-written one always lacks. + * Throwaway evidence, deleted at S8 (`PLAN.md` → the sunset rule). What it + * checks, why, and every planted control: see `README.md` beside this file. * - * **A totality assertion.** Every `.mts` file under `goals/devforce-lab/` is - * classified as lab code, a goal runner, or a fixture. A checker that only - * inspects the files it already knew about cannot report the one nobody listed. - * - * **A negative control.** `--plant ` injects the exact defect a claim - * exists to catch, so the green can be watched going red before it is believed. - * - * Throwaway evidence, not production code. Nothing imports it, it is not a - * workspace package, and it is not in any build, test or lint discovery path. - * - * Run: node specs/issues/FIX-1496/poc/gap-check/check.mjs - * Control: node specs/issues/FIX-1496/poc/gap-check/check.mjs --plant unclassified-file - * node specs/issues/FIX-1496/poc/gap-check/check.mjs --plant channel-is-opened - * node specs/issues/FIX-1496/poc/gap-check/check.mjs --plant durable-stores - * node specs/issues/FIX-1496/poc/gap-check/check.mjs --plant model-check-passed - * node specs/issues/FIX-1496/poc/gap-check/check.mjs --plant list + * Run: node specs/issues/FIX-1496/poc/gap-check/check.mjs [--plant |list] */ import { readFileSync, readdirSync, statSync } from "node:fs"; import { join, relative } from "node:path"; @@ -34,11 +15,30 @@ const REPO = fileURLToPath(new URL("../../../../../", import.meta.url)); const LAB = join(REPO, "goals/devforce-lab"); const SIBLINGS = ["goals/pentest-lab", "goals/manager-queue-lab"]; +/** + * The browsable child-session surface FIX-1440 removes. + * + * Case-insensitive and anchored on the canonical client method + * (`listChildSessions`, `packages/client/src/session-client/sessions.ts`) and + * its types. The first version of this pattern spelled `childSessions` with a + * lower-case c and therefore could not match `listChildSessions` — the exact + * call it existed to catch. Found by reproduction, not by reading, which is why + * `browse-surface` below is now a planted control. + */ +const BROWSE_SURFACE = /\/children\b|childsession|listchildren/i; + +/** Dispatch parentage read as provenance — the form FIX-1440 explicitly keeps. */ +const PROVENANCE = /parentage:\s*\{\s*parentOf/; + const PLANTS = [ "unclassified-file", "channel-is-opened", "durable-stores", "model-check-passed", + "browse-surface", + "provenance-gone", + "sibling-lost-channel", + "artifact-pushed", ]; const plant = (() => { @@ -71,41 +71,43 @@ const failures = []; const evidence = []; const claim = (ok, red, green) => (ok ? evidence.push(green) : failures.push(red)); -// The tree, with the plant applied to the in-memory copy only. Nothing on disk -// is touched, so a control run cannot leave the repository perturbed. +// Plants are applied to the in-memory copy only; nothing on disk is touched, so +// a control run cannot leave the repository perturbed. let tree = filesUnder(LAB); +const patch = (match, fn) => { + tree = tree.map(([p, t]) => (p.includes(match) ? [p, fn(t)] : [p, t])); +}; + if (plant === "unclassified-file") { tree = [...tree, ["goals/devforce-lab/lab/workforce/flows/helpers.ts", "// planted\n"]]; } if (plant === "channel-is-opened") { - tree = tree.map(([p, t]) => - p.endsWith("lab/host.mts") ? [p, `${t}\n// planted: await openChannels(channels, {});\n`] : [p, t], - ); + patch("lab/host.mts", (t) => `${t}\n// planted: await openChannels(channels, {});\n`); } if (plant === "durable-stores") { - tree = tree.map(([p, t]) => - p.includes("it-commits-from-the-seats-own-file") - ? [p, t.replace(/inMemoryStores\(\)/g, "sqliteStores()")] - : [p, t], - ); + patch("it-commits-from-the-seats-own-file", (t) => t.replace(/inMemoryStores\(\)/g, "sqliteStores()")); } if (plant === "model-check-passed") { - tree = tree.map(([p, t]) => - p.endsWith("it-commits-from-the-seats-own-file/goal.md") - ? [p, t.replace("| NOT RUN |", "| PASS |")] - : [p, t], - ); + patch("it-commits-from-the-seats-own-file/goal.md", (t) => t.replace("| NOT RUN |", "| PASS |")); } +if (plant === "browse-surface") { + // The canonical call, spelled exactly as the client exports it. This is the + // case the first pattern missed while reporting green. + patch("lab/host.mts", (t) => `${t}\nconst rows = await client.sessions.listChildSessions(id);\n`); +} +if (plant === "provenance-gone") { + patch("lab/host.mts", (t) => t.replace(/parentage:\s*\{\s*parentOf/, "filter: { startedBy")); +} +if (plant === "artifact-pushed") { + patch("lab/scratch-repo.mts", (t) => `${t}\n// planted\nexport const publish = (d) => git("push", "origin", d);\n`); +} + +let siblingDirs = SIBLINGS; +if (plant === "sibling-lost-channel") siblingDirs = [...SIBLINGS, "goals/task-board"]; -const text = (suffix) => tree.filter(([p]) => p.endsWith(suffix)); -const body = (suffix) => text(suffix).map(([, t]) => t).join("\n"); +const body = (suffix) => tree.filter(([p]) => p.endsWith(suffix)).map(([, t]) => t).join("\n"); -// --------------------------------------------------------------------------- -// Claim 0 — totality. Every .mts under the lab is one of three known kinds. -// -// This runs first and its failure is fatal: every claim below reads a subset of -// these files, and a file none of them classified is a file none of them saw. -// --------------------------------------------------------------------------- +// --- Claim 0 — totality. Fatal: every claim below reads a subset of these. --- { const mts = tree.filter(([p]) => p.endsWith(".mts") || p.endsWith(".ts")); const unclassified = mts.filter(([p]) => { @@ -126,10 +128,7 @@ const body = (suffix) => text(suffix).map(([, t]) => t).join("\n"); } } -// --------------------------------------------------------------------------- -// Claim 1 — two goal checks, exactly one of them model-backed, and the -// model-backed one has never recorded a passing run. -// --------------------------------------------------------------------------- +// --- Claim 1 — two goal checks, one model-backed, never a passing run. --- { const goals = tree.filter(([p]) => p.endsWith("/goal.md")).map(([p]) => p); claim( @@ -146,10 +145,8 @@ const body = (suffix) => text(suffix).map(([, t]) => t).join("\n"); ); const honesty = tree.find(([p]) => p.endsWith("it-commits-from-the-seats-own-file/goal.md")); - const verdicts = (honesty?.[1] ?? "") - .split("\n") - .filter((line) => /^\|\s*\d{4}-\d{2}-\d{2}/.test(line)); - const passes = verdicts.filter((line) => /\|\s*PASS/.test(line)); + const verdicts = (honesty?.[1] ?? "").split("\n").filter((l) => /^\|\s*\d{4}-\d{2}-\d{2}/.test(l)); + const passes = verdicts.filter((l) => /\|\s*PASS/.test(l)); claim( verdicts.length > 0 && passes.length === 0, `the model-backed check records ${passes.length} passing verdict(s); the spec's whole ` + @@ -158,10 +155,7 @@ const body = (suffix) => text(suffix).map(([, t]) => t).join("\n"); ); } -// --------------------------------------------------------------------------- -// Claim 2 — the artifact does not outlive the run: the repository it is -// committed into is created under the OS temp directory. -// --------------------------------------------------------------------------- +// --- Claim 2 — the artifact does not outlive the run. --- { const scratch = body("lab/scratch-repo.mts"); claim( @@ -173,32 +167,23 @@ const body = (suffix) => text(suffix).map(([, t]) => t).join("\n"); ); claim( !/git\(\s*"push"/.test(scratch) && !/\bgh\b/.test(scratch), - `the scratch repository now pushes or shells to gh, so the artifact may already leave the run`, + `the artifact's repository helper now pushes or calls gh, so the artifact may already ` + + `leave the run`, `nothing in the artifact's repository helper pushes or calls gh`, ); } -// --------------------------------------------------------------------------- -// Claim 3 — the model-backed check runs on stores that do not survive it. -// --------------------------------------------------------------------------- +// --- Claim 3 — the model-backed check runs on stores that do not survive it. --- { - const honesty = body("it-commits-from-the-seats-own-file/run.mts"); claim( - /inMemoryStores\(\)/.test(honesty), + /inMemoryStores\(\)/.test(body("it-commits-from-the-seats-own-file/run.mts")), `the model-backed check no longer runs on inMemoryStores(), so ER-3's "durable across the ` + `run" may already be met`, `the model-backed check runs on inMemoryStores(), which ER-3 calls not durable`, ); } -// --------------------------------------------------------------------------- -// Claim 4 — the declared channel is walked but never opened, while the two -// sibling labs that prove the path both open theirs. -// -// The positive half is what makes this a gap rather than a missing primitive: -// a claim that only said "devforce does not open a channel" is equally true of -// a framework with no channels in it. -// --------------------------------------------------------------------------- +// --- Claim 4 — the channel is walked but never opened, while siblings open theirs. --- { const opensHere = tree.filter(([, t]) => /\bopenChannels\s*\(/.test(t)).map(([p]) => p); claim( @@ -215,37 +200,36 @@ const body = (suffix) => text(suffix).map(([, t]) => t).join("\n"); `the devforce lab declares ${declared.length} channel file(s) that nothing opens`, ); - const siblingsThatOpen = SIBLINGS.filter((dir) => + // The positive half. Without it, "devforce does not open a channel" is + // equally true of a framework that has no channels at all. + const open = siblingDirs.filter((dir) => filesUnder(join(REPO, dir)).some(([, t]) => /\bopenChannels\s*\(/.test(t)), ); claim( - siblingsThatOpen.length === SIBLINGS.length, - `only ${siblingsThatOpen.length} of ${SIBLINGS.length} sibling labs open a channel, so ` + - `"the path is already proven next door" is weaker than the spec claims`, - `both sibling labs (${SIBLINGS.join(", ")}) open a channel on the same primitive`, + open.length === siblingDirs.length, + `only ${open.length} of ${siblingDirs.length} sibling labs open a channel, so "the path is ` + + `already proven next door" is weaker than the spec claims: ` + + `${siblingDirs.filter((d) => !open.includes(d)).join(", ")} does not`, + `all ${siblingDirs.length} sibling labs open a channel on the same primitive`, ); } -// --------------------------------------------------------------------------- -// Claim 5 — nothing in the lab reads the browsable child-session surface -// FIX-1440 removes, so an unlanded FIX-1440 does not fence this work. -// -// `parentage` is provenance, which FIX-1440's own owner amendment keeps. -// --------------------------------------------------------------------------- +// --- Claim 5 — FIX-1440 does not fence this work. --- { - const browse = tree.filter(([, t]) => /\/children\b|childSessions|listChildren/.test(t)); + const browse = tree.filter(([, t]) => BROWSE_SURFACE.test(t)); claim( browse.length === 0, - `the lab reads the child-session browse surface FIX-1440 removes (${browse - .map(([p]) => p) - .join(", ")}), so this work IS fenced by FIX-1440`, - `nothing in the lab reads the browsable child-session surface FIX-1440 removes`, + `the lab reads the child-session browse surface FIX-1440 removes ` + + `(${browse.map(([p]) => p).join(", ")}), so this work IS fenced by FIX-1440`, + `nothing in the lab reads the browsable child-session surface FIX-1440 removes ` + + `(pattern covers the canonical listChildSessions)`, ); - const provenance = tree.filter(([, t]) => /parentage:\s*\{\s*parentOf/.test(t)); + const provenance = tree.filter(([, t]) => PROVENANCE.test(t)); claim( provenance.length >= 1, - `the lab no longer reads dispatch parentage at all, so the FIX-1440 finding needs re-deriving`, + `the lab no longer reads dispatch parentage as provenance, so the FIX-1440 finding needs ` + + `re-deriving rather than carrying forward`, `the lab reads dispatch parentage as provenance only (${provenance.length} site), which ` + `FIX-1440's owner amendment explicitly keeps`, ); From 853c75d50b840748884be0ef428e247dbf88cb18 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 22 Sep 2026 02:38:53 +0000 Subject: [PATCH 3/3] spec(FIX-1496): restate D1 to the enacted one-leg cut MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit D1's heading still described the pre-trim shape ("a pull request on a separate throwaway repository") while PLAN.md S3, BR-14/BR-15 and the Decided-not-asked item all carried the enacted cut. The decision is unchanged; the document disagreed with its own plan. The heading, the Instead-of / Because / Locks-in rows and the decision tree node now state it: work at an address that outlives the run — a pushed bare clone on the one automated leg, a pull request for the release run. The three falsifiable properties are untouched. Two consequences of the same drift, swept while in there: - SPEC.md's sign-off line 1 mirrored the old heading, and a card that changed beside a sign-off that did not is the same defect. Also its people table ("the pull request's address" in a verdict log that on the automated leg holds the clone address) and the figure's alt text. - The brief's diff told the seat to put the marker in a pull request body, which no longer exists on the leg CI runs. It is now the commit message, which is true on both. "What would change my mind" is restated rather than kept verbatim: half of it already fired. It said a bare repository at a declared path would be cheaper if "outside the run" meant outside the process — and round 1 took exactly that for the automated leg. What remains open is whether the release proof also settles for the clone, with a recommendation against and the reason. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016qaz1VpRK4wCQzxpjzZ4Hz --- specs/issues/FIX-1496/DECISIONS.md | 27 ++++++++++++++++++--------- specs/issues/FIX-1496/SPEC.md | 17 +++++++++-------- 2 files changed, 27 insertions(+), 17 deletions(-) diff --git a/specs/issues/FIX-1496/DECISIONS.md b/specs/issues/FIX-1496/DECISIONS.md index 474e130136..83e9ecbd50 100644 --- a/specs/issues/FIX-1496/DECISIONS.md +++ b/specs/issues/FIX-1496/DECISIONS.md @@ -9,8 +9,8 @@ sign-off surface; everything else here is context for them. ```mermaid flowchart TD - I["FIX-1496 · which artifact counts, and how thin"] --> D1["D1 · a pull request on a separate repo
real = three falsifiable properties"] - D1 -.->|"rejected"| X1["a commit in a durable local repo
only the runner can reach it · closer to a transcript"] + I["FIX-1496 · which artifact counts, and how thin"] --> D1["D1 · work at an address that outlives the run
bare clone in CI, a pull request for the release run
real = three falsifiable properties"] + D1 -.->|"rejected"| X1["a commit left in the run's temp directory
swept away · a transcript with a SHA"] D1 -.->|"rejected"| X2["a pull request on this repository
strongest evidence, but an agent opening PRs on production"] I --> D2["D2 · grade the product against a
pre-stated, executable condition"] D2 -.->|"rejected"| X3["keep grading only the prompt
proves the plumbing, never the work"] @@ -23,13 +23,13 @@ is **not** a fork: driving the channel is the epic's wording, recorded under [Decided, not asked](#decided-not-asked). -## D1 · The artifact is a pull request on a separate throwaway repository, and "real" is defined by three falsifiable properties rather than by taste +## D1 · The artifact is work left at an address that outlives the run — a pushed bare clone on the one automated leg, a pull request for the release run — and "real" is three falsifiable properties rather than a judgement call | | | |---|---| -| **Instead of** | A commit in a durable local repository, or a pull request against `flow-state-dev` itself | -| **Because** | ER-DevForce contrasts a real artifact with *a transcript*. A commit only the machine that produced it can reach is a transcript with a SHA. A pull request has an address a person opens, a title and a body the run wrote, and a diff a person accepts or rejects — which is the thing the launch claim is actually about | -| **Locks in** | Producing the release proof costs a credential, a network call and a throwaway repository, once. And we have committed to a definition of *real work* that the next Lab proof has to keep meeting — a later artifact that cannot satisfy the three properties does not get to call itself real by analogy | +| **Instead of** | Leaving the commit in the run's own temporary directory, as today; or opening the pull request against `flow-state-dev` itself | +| **Because** | ER-DevForce contrasts a real artifact with *a transcript*. What makes something more than a transcript is that it is still there, at an address, once the run is over — so the automated leg pushes to a **bare clone at a declared path** rather than leaving a commit in a directory the machine will sweep. The release run points that same path at a real remote and opens a pull request, because a title, a body and a diff a person accepts or rejects is what the launch claim is actually about | +| **Locks in** | The **release** run costs a credential, a network call and a throwaway repository, once — CI costs none of them, which is what the one-leg trim bought. And we have committed to a definition of *real work* that the next Lab proof has to keep meeting: a later artifact that cannot satisfy the three properties does not get to call itself real by analogy | **The three properties, each with its falsifier.** This is what makes *"would a person accept this as real work?"* checkable instead of a matter of taste: @@ -40,9 +40,18 @@ this as real work?"* checkable instead of a matter of taste: | **The Lab did not author it** | Any graded token of the artifact's content is also found in the lab's own code, its fixtures, or the prompt — the held-out discipline, inverted onto the product | | **It satisfies a condition the requester stated before the run, executed** | The condition is run in the produced tree and does not pass; or the condition would have passed before the run, which makes it no evidence at all | -**What would change my mind:** if the owner reads "exists outside the run" as *outside the -process* rather than *outside the machine*, a bare repository at a declared path is cheaper and -needs no credential. Say so and the pull-request leg becomes optional rather than the proof. +**What would change my mind — and half of it already has.** The original card said that if +*"exists outside the run"* means outside the **process** rather than outside the **machine**, +then a bare repository at a declared path is cheaper and needs no credential. Review round 1 +took exactly that for the automated leg, which is why CI now needs no token: the cheap reading +won everywhere it could. + +What is still open is the other half: **whether the release proof also settles for the bare +clone.** If the owner reads a durable local address as enough evidence for a launch claim, the +pull-request leg drops entirely and this decision reduces to the three properties with no remote +at all. I would not recommend it — a person outside the run cannot open a path on somebody's +disk, and that is the audience the whole proof exists for — but it is a one-line change to make, +and nothing is built against it. ## D2 · The proof grades what the run produced, against an acceptance condition the brief states first and a machine executes diff --git a/specs/issues/FIX-1496/SPEC.md b/specs/issues/FIX-1496/SPEC.md index 399ae8e703..161120f582 100644 --- a/specs/issues/FIX-1496/SPEC.md +++ b/specs/issues/FIX-1496/SPEC.md @@ -8,8 +8,8 @@ Feature · `goals/devforce-lab` only · **small** · 1 PR · epic [FIX-1457](../ | Someone who… | Today | After | |---|---|---| -| **is deciding whether Workforce is ready to ship** | "A Lab can build" rests on a check that was written, type-checked, and **never executed**. Its verdict log has one row and that row reads `NOT RUN` | Opens a pull request a hired coder seat wrote, and reads the diff. The verdict log carries a dated `PASS` with the pull request's address in it | -| **asks what the workforce actually produced** | A commit in a temporary directory on whatever machine ran it, swept away with the rest of `/tmp` | A pull request with a title, a body and a diff, at an address that still resolves after the run is over | +| **is deciding whether Workforce is ready to ship** | "A Lab can build" rests on a check that was written, type-checked, and **never executed**. Its verdict log has one row and that row reads `NOT RUN` | Opens the work a hired coder seat produced and reads the diff. The verdict log carries a dated `PASS` naming the artifact's address and which leg ran | +| **asks what the workforce actually produced** | A commit in a temporary directory on whatever machine ran it, swept away with the rest of `/tmp` | Work at an address that still resolves after the run is over — a pushed branch in CI, and for the release run a pull request with a title, a body and a diff | | **asks whether the work is any good, not merely that it exists** | Nothing grades the product. The check grades the *prompt*, deliberately, so a model that improvised a plausible file still passes | The brief names an acceptance condition **before** the run, and the check executes it against what the run produced. Improvising does not pass | | **asks whether the seats really used their channels** | The feature channel is a Markdown file the loader walks and nothing ever opens | A post on the feature channel is what starts the work. The channel is driven, not decorated | | **has to re-run this proof a year from now** | Impossible: it has never been run once | One command, and it is what CI runs: no network, no token, no `gh`. The pull-request release run is the same path pointed at a real remote | @@ -22,7 +22,7 @@ and letting what it makes survive. ## What changes -![Two lanes over the same five-stage DevForce path. Today three stages are built and green while the declared channel sits unused and the artifact is a commit in a temp directory; a banner reads never run, on in-memory stores. After, the same five stages with four marked gaps closed: a post on the feature channel drives the filing, the run is on durable on-disk stores, the artifact is a pull request at an address a person can open, and it is graded against an acceptance condition the brief stated before the run.](figures/the-four-gaps.svg) +![Two lanes over the same five-stage DevForce path. Today three stages are built and green while the declared channel sits unused and the artifact is a commit in a temp directory; a banner reads never run, on in-memory stores. After, the same five stages with four marked gaps closed: a post on the feature channel drives the filing, the run is on durable on-disk stores, the artifact is work left at an address that outlives the run, and it is graded against an acceptance condition the brief stated before the run rather than assumed.](figures/the-four-gaps.svg) The five stages are the same in both lanes, and the middle three are untouched. Only the two ends move, plus two properties of the run. **Nothing new is built** — the four gaps are the whole @@ -48,7 +48,7 @@ issue ([D1](DECISIONS.md#d1), [D2](DECISIONS.md#d2), and the channel leg the epi + checkout and you cannot edit it. Adding a test of your own does not + substitute for it. + -+ Put this marker in the body of the pull request you open: ++ Put this marker in your commit message: FEATURE-BRIEF-E61B8 ``` @@ -83,10 +83,11 @@ declared and never dispatched, which is a negative claim the existing gate alrea ## Sign off -1. **[D1](DECISIONS.md#d1) · The artifact is a pull request on a separate throwaway repository, - and "real" is three falsifiable properties rather than a judgement call.** *If wrong:* the - release proof needs a token and network once, and we have committed to a definition of "real - work" that a later Lab has to keep meeting. +1. **[D1](DECISIONS.md#d1) · The artifact is work left at an address that outlives the run — a + pushed bare clone in CI, a pull request for the release run — and "real" is three falsifiable + properties rather than a judgement call.** *If wrong:* the release run needs a token and + network once, and we have committed to a definition of "real work" that a later Lab has to + keep meeting. 2. **[D2](DECISIONS.md#d2) · The proof grades what the run produced, against a condition the brief stated first and a machine executes.** *If wrong:* every future DevForce brief owes an executable acceptance condition, and a brief that cannot state one cannot be proved this way.