Skip to content

feat(harness-manager): FIX-1762 provision runs through the workspace host (3/4) - #2744

Merged
jhoffner merged 5 commits into
claude/project-thread-6n4tdxfrom
claude/fix-1762-harness-manager-cztr0w
Oct 5, 2026
Merged

jhoffner merged 5 commits into
claude/project-thread-6n4tdxfrom
claude/fix-1762-harness-manager-cztr0w

Conversation

@jhoffner

@jhoffner jhoffner commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

Requested by Jake · project thread

PR 3 of 4 for FIX-1762, stacked on #2737 (based on its head e66372b45). Spec: specs/issues/FIX-1762/, PLAN row 3 (S4).

Before

harness-manager made its own checkouts. It had its own process runner, its own git worktree provisioning, its own completion detection, its own "same repository" check, and its own lock. That was a second copy of what the workspace host from #2737 does. A run could only come from one fixed sourceRepo, and nothing a run wrote outside git was saved.

After

  • harnessManager({ workspace }) takes a workspace host, or the fixed { root, sourceRepo, baseRef } as before. Both go through one provisioning path. A fixed sourceRepo becomes a host whose source always answers that repository. Checkout paths and branch names are unchanged, and the derivation snapshot still passes.
  • A repository on this machine is used directly. localWorkspaceHost({ localRepositories }) cuts the run's branch in that repository itself. There is no clone, no fetch on a retry, and the branch stays in sourceRepo, as it did before. The allowlist is still checked before any git runs, and any other bare path is still refused.
  • The run record keeps the remote and base from the first provision. A retry uses them even after the source moves (BR-19). A run that started with no repository is recorded as one (filesOnly) and stays on its kept files if the project gains a repository.
  • Kept files are saved at the end of each turn, when the run parks on a question, and when the harness fails. The result goes on the record's lastSave. A failed save doesn't fail the run, except at completion: a run whose phase is done but whose save failed fails that attempt, so the retry saves the files.
  • A provision's budget covers its lock waits. Time queued behind another provision of the same place or clone counts against provisionTimeoutMs. Each provision of a files place holds it under a fresh token, so a superseded attempt's handle can't save or release its replacement's files.
  • A run its workspace refuses is not retried. That covers a source answering refused and a host refusing what the source named. The manager throws HarnessRunRefused, and its failure recorder cancels the row on its first attempt, under that attempt's claim, with the refusal as the reason. No retry is granted, so the refusal can't burn through the row's attempts.
  • In a run with no repository, the manager's question marker isn't saved into the project's collection. The host leaves a place request's ignored directory out of a files run's save.
  • No git or child process runs from harness-manager. A fence test fails if child_process or execFile comes back into src/.
  • harness-manager does not import Workforce.

How

  • workspace:
    • New worktree.ts holds the hardened provisioning. Both the clone path and the local path use it: one budget for the whole provision, the interrupted-provision marker checked against the tree, refusals for a deleted, switched or foreign branch, and an optional ignored directory the repository must keep out of git.
    • New repository.ts holds the single identityFromCommonDir, plus repositoryIdentity and resolvesToCommit.
    • lock.ts also serves a run-long lease with no heartbeat (signal, injectable clock).
    • The host gains locate, provisionTimeoutMs and localRepositories.
    • For a files run, the host hides ignored.dir inside workspace/ from the flush's listing, so save never writes it.
  • harness-manager:
    • src/exec.ts is deleted.
    • Its completion detection and identity check are deleted. checkout.ts and guards.ts re-export the workspace versions.
    • The run-long lease (obligation B) is now a thin wrapper over workspace's acquireLock.
    • manager.ts asks the host's source and pins the answer, provisions through the host, records the remote and base on the first provision, and saves at the three save points.
    • A refusal (a source refused answer, or a WorkspaceRefusedError from provision) becomes HarnessRunRefused. recordFailure cancels the row with a claim ticket for the attempt, so a superseded attempt can't cancel its replacement's row.
  • Docs:
    • Both package READMEs are updated.
    • The harness-manager limits section on the docs site is updated.
    • A minor changeset covers harness-manager and workspace.

Verification

  • V4 (BR-22): the existing harness-manager suite passes through the host: 12 files, 210 tests, including workspace.spec, message-door, mailbox-board and slot. The workspace suite passes: 176 tests. labs/conductor passes 154 tests. Typecheck passes for workspace, harness-manager, labs/conductor and goals.
  • New test/workspace-host.spec.ts drives the full board → manager → host flow. Each check below goes red when its mechanism is removed:
    • Saved at turn end and on park: red without the save in decide.
    • Saved on harness failure: red without the save in recordFailure.
    • BR-19 retry stays on remote A after the source moves to B: red without pinnedAnswer.
    • A source refusal and a host refusal (remote not allowed), with maxAttempts: 3: the row is cancelled at attempt 1 with no retry granted, the harness never runs, and a later drain changes nothing. Red without the cancel in recordFailure, which re-pends with a retry granted.
    • A parked files run saves only its work, not .fsdev/ask/…. Red without the host's listing filter. A matching unit test is in local-host-files.spec.ts.
  • Review fixes (Codex, 5 findings). Each has a test that fails without its fix: a files run stays one after a repository is added, a final save that fails doesn't complete, a lock wait fails inside the budget, a replaced place handle can't save, and a relative sourceRepo provisions.
  • New test/local-host-local.spec.ts (workspace) shows the branch is cut in the local repository with no clone and no fetch on retry, and that the ignored-directory refusal leaves nothing behind.
  • Negative control for the local-repository path: forcing localRepository() to answer nothing (the remote path) fails 6 of 8 local-host-local tests and 35 harness-manager workspace.spec tests.

What's asked of you

One call to check: a refused run settles as cancelled, not errored. The task substrate has no "fail without retry" write, since fail re-pends while attempts remain. Adding one would change orchestration, so I didn't. If you'd rather it read errored, that's a small substrate change for a follow-up.

A note for PR 4: a phase's validate receives the host when a host is passed, so devforce-lab's phase.mts needs to stop reading sourceRepo there.

🤖 Generated with Claude Code

https://claude.ai/code/session_01P9UuEKLJUWaKiSrYWCJbt5
https://claude.ai/code/session_011KTy6FK6trgHQhStU5yxgi

claude added 4 commits October 4, 2026 22:24
…e worktree provisioning (FIX-1762)

A host lists repositories on its own machine in `localRepositories`. A
source answering one of those paths gets a worktree of that repository
itself: no clone, no fetch on a retry, and the run's branch kept in it.
Any other bare path is still refused before git runs.

Worktree provisioning moves into `worktree.ts` and is shared by the
clone path and the local path: one budget for the whole provision,
the interrupted-provision marker corroborated against the tree,
refusals for a deleted, switched or foreign branch, and an optional
directory the repository must keep out of git.

Exports what harness-manager needs to stop keeping its own copies:
the "same repository" check, the bounded process runner and the lock.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P9UuEKLJUWaKiSrYWCJbt5
…-1762)

`harnessManager({ workspace })` takes a workspace host as well as a
fixed `{ root, sourceRepo, baseRef }`, which becomes a host whose
source always answers that repository. One provisioning path either
way; the checkout paths and branches a fixed repository derives are
unchanged.

The run record keeps the remote and base a run was first provisioned
from, and a retry uses them after the source moves (BR-19). Kept files
are saved at the end of each turn, when the run parks, and when the
harness fails, with the result on the record's `lastSave`.

harness-manager's own exec, completion detection and identity check
are deleted in favour of the host's; its run-long lease is a thin
wrapper over the host's lock. No git runs from this package any more.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P9UuEKLJUWaKiSrYWCJbt5
…les run's save (FIX-1762)

A run with no repository works inside its kept files, so a caller's own
file there (harness-manager's question marker) was saved into the
project's collection with the run's work. The host now hides
`ignored.dir` inside `workspace/` from the flush's listing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P9UuEKLJUWaKiSrYWCJbt5
…etrying it (FIX-1762)

A source answering `refused`, or a host refusing what the source named,
gives the same answer on every retry, so failing the attempt spent the
row's attempts one by one on it. The manager now throws
`HarnessRunRefused`, and its failure recorder cancels the row with the
refusal as the reason, under the attempt's own claim. The board's
recorder then has nothing to re-queue: one attempt, no retry granted.

Also pins that a files run's question marker is not saved with its
work.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P9UuEKLJUWaKiSrYWCJbt5
@jhoffner jhoffner self-assigned this Oct 4, 2026
@vercel

vercel Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
flow-state-dev-docs Ignored Ignored Oct 4, 2026 10:43pm UTC
flow-state-dev-kitchen-sink Ignored Ignored Oct 4, 2026 10:43pm UTC
flow-state-dev-ui Ignored Ignored Preview Oct 4, 2026 10:43pm UTC

Request Review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-04T22:33:05.926802Z 74cd0f7 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cursor

cursor Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 4997f70f-c588-4000-8bc0-59624fae4f16)

@linear-code

linear-code Bot commented Oct 4, 2026

Copy link
Copy Markdown

FIX-1762

jhoffner commented Oct 4, 2026

Copy link
Copy Markdown
Collaborator Author

Second look + architecture pass (read-only; for the implementing agent to triage)

Overall: The shape is right. One provisioning path, the git moved into workspace, exec.ts deleted, and a fence test. Replacing the fixed-repo config with hostForConfig is leaner than a parallel code path would have been. The notes below are mostly small. Only 1 and 2 are worth acting on in this PR. I read the diff and did not run the suites.

Worth doing in this PR (they touch code this PR already rewrites)

  1. Dead ternary in manager.ts. provisionTimeoutMs: isWorkspaceHost(workspace) ? workspace.provisionTimeoutMs : workspace.provisionTimeoutMs has identical branches. Either WorkspaceConfig and WorkspaceHost both expose it, so drop the ternary, or the intent was something else, such as reading it from host. Since host is always built, host.provisionTimeoutMs is the single read and isWorkspaceHost needs no second use there.

  2. pinnedAnswer re-implements RunSourceAnswer internals in the manager. It rebuilds repo/files answers field by field, so any new answer field (or a new kind) is silently dropped from a pinned retry. That is the BR-19 path, so a regression would be quiet. The pin is a workspace concept ("provision from this remote/base, keep the source's files"). I'd move it into @flow-state-dev/workspace, either as a pin?: { remote, baseRef } option on provision or as a helper exported beside RunSourceAnswer in run-source.ts. The manager then passes the recorded values and never constructs an answer. This also shrinks manager.ts, which is already 2,095 lines and gained ~230 more.

Worth considering (judgment calls)

  1. The source is asked twice per attempt, with a drift check between. prepare calls host.source(ctx) and provision calls it again, and a changed cwd throws a retryable HarnessAttemptFailed. The reason (a collection ref is bound to its context) is sound. But if the source can move between the two calls, the drift check burns an attempt on something that will repeat. Is a run whose source answers differently each call a refusal rather than a retry? At minimum, say in the comment which one it is.

  2. recordFailure swallows a failed cancel. Then the board recorder re-pends and the run is retried, which is the behavior this PR set out to remove. The catch {} is defensible for a best-effort write, but a refusal that silently falls back to retry should at least leave a trace, such as a log line or the reason on the run record. A test where tasks.cancel rejects would pin down which behavior you want.

  3. cancelled vs errored (the question you asked). I agree with not changing orchestration here. A refusal is a policy outcome, and cancelled with a reason is honest for now. I would file the follow-up for a "settle without retry" write now, because every consumer reading cancelled as "someone stopped this" (conductor UI, metrics) will misread refusals. The reason string is the only signal, and it is prose.

Architecture (only where this PR is already the opportunity)

  1. harness-manager/src/workspace.ts is now two modules in one file. What remains is the identity derivation (encodeSegment, assertSafeSegment, locationSegments, branchFor, placeFor), the config-to-host adapter (hostForConfig, provisionCheckout) and the lease (acquireCheckout). Its header still says "Provisioning… the git itself is the workspace host's", and the 40-line identity-rule comment now explains a module that does less. Since provisionCheckout is only reached from checkout.ts and tests, I'd leave the file split for a later change. In this PR, trim the header so it matches what the file does.

  2. provisionCheckout and checkoutPathFor are now test/lab-only. manager.ts uses hostForConfig + placeFor + host.locate. checkoutPathFor stays only as the parity oracle for the derivation snapshot, which is a good reason to keep it. Add a one-line comment saying so, so nobody "simplifies" it away and loses the snapshot's independence.

  3. phase.validate(workspace | host) widens a public type for a one-consumer problem, and your PR note already says devforce-lab must stop reading sourceRepo in PR 4. Fine in a stack. Confirm PR 4 lands in the same release, or the minor changeset ships a signature that breaks existing validate implementations typed against WorkspaceConfig.

Not flagged

Run-record fields (remote, baseRef, lastSave) use .nullable().default(null), as BP-023/BP-030 want. Save points, the refusal path, and the matching red-state tests look well covered.


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Snob — already tight. No POC.

Head 74cd0f75dcde38d67193ac30f2861c0a8283a1f2. I read manager.ts (pinnedAnswer, saveKeptFiles, the host wiring), workspace.ts (hostForConfig, provisionCheckout, acquireCheckout), timeouts.ts, checkout.ts, guards.ts, worktree.ts, local-host.ts (provisionLocalCheckout, provisionClonedCheckout, leavingOut), repository.ts, lock.ts, and workspace-host.spec.ts.

The move into provisionWorktree is the subtraction. The local path and the clone path are the two adapters over it (lock, marker, checkout path, fetch versus HEAD). One function with a mode is as long as that difference. pinnedAnswer's two file branches are the same extraction; collapsing them is a few lines, not a shape. A helper around the two source asks is longer than the check it replaces, and the second ask stays because a collection ref is bound to the context that resolved it. saveKeptFiles is the one save; the failure path's .catch is what keeps a save error from replacing the error being unwound. timeouts.ts is the one wait that is not a git budget. acquireCheckout and provisionCheckout are the checkout entry conductor and the workspace tests call. leavingOut inlined ties. The provisionTimeoutMs ternary reads the same property on both sides of isWorkspaceHost; one line, not a rewrite.

Not reopening the cancelled-versus-errored settlement, or the phase validate note left for PR 4.

Open in Web View Automation 

Sent by Cursor Automation: Code Snob

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment

FIX-1762 moves git/exec/worktree provisioning into @flow-state-dev/workspace and teaches harnessManager to accept a WorkspaceHost (or the existing fixed { root, sourceRepo, baseRef } via hostForConfig). The main simplification question is whether that seam added avoidable complexity.

Verdict: the approach is sound and I’m approving. The old harness-local git stack was the wrong layer; centralizing run, locks, worktree.ts, and localWorkspaceHost matches the workspace package charter and gives other workers a shared path. What stays in harness-manager (path/branch grammar, attempt-scoped checkout lease, pin remote/baseRef on the run record, kept-files save at stop points, HarnessRunRefused + cancel-without-retry) is product logic that would land in the manager under any design.

Complexity mostly moved (~800 lines out of harness workspace.ts) rather than net-new. The tradeoff is two heavy files (manager.ts orchestration + local-host.ts) instead of one monolith — acceptable for a 3/4 stack PR.

Simpler alternatives (considered, not blocking)

Alternative Why it wasn’t taken / why I’m not pushing it
Host-only API — drop WorkspaceConfig from options; callers always pass hostForConfig(...) Shrinks isWorkspaceHost and union typing; pushes boilerplate to every fixed-repo mount. Reasonable follow-up, not clearly simpler overall.
Push save/refusal into workspace Would mix run-record and board semantics into the host; current layering keeps workspace about places and projections.
simple-git / execa / proper-lockfile Provisioning markers, single cross-command budget, stale-lock re-read, and ask-marker ignore rules are bespoke; npm wrappers don’t remove the hard parts and fight the “one run module” convention.

Performance

No busy-waits or logging in hot paths. Save/flush does sync tree walks + per-file reads (security-boundary tradeoff); remote provision refreshes on new branch cuts by design. Fine at typical board scale.

Optional polish before or right after merge

See inline notes on the dead provisionTimeoutMs ternary and the duplicated accessor comment blocks — quick wins, not architectural.

Non-blocking follow-ups

  • Harden or document isWorkspaceHost (provision duck check) if WorkspaceConfig ever grows.
  • Make pinnedAnswer easier to audit (switch vs nested ternary) for BR-19.
  • Cross-package test import of workspace/test/fake-collection — consider a shared testing export.
  • host.source(ctx) twice per attempt is fine if sources stay cheap; memoize per attempt if they won’t.

No changeset concerns from this review; the PR already carries the appropriate fragment.

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

Comment thread packages/harness-manager/src/manager.ts Resolved
Comment thread packages/harness-manager/src/manager.ts Resolved
Comment thread packages/harness-manager/src/manager.ts Resolved
Comment thread packages/harness-manager/src/manager.ts Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 74cd0f75dc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/harness-manager/src/manager.ts Resolved
Comment thread packages/harness-manager/src/manager.ts Resolved
Comment thread packages/harness-manager/src/manager.ts Resolved
Comment thread packages/workspace/src/local-host.ts Resolved
…ing and saves (FIX-1762)

- A run first provisioned with no repository is recorded as one
  (`filesOnly`), so a project that gains a repository before a retry
  does not move it off the files it saved.
- A provision's budget covers its waits for the place and clone locks,
  in process and on disk, so a caller sizing its lease from
  `provisionTimeoutMs` is not outlived by a queued provision.
- Each provision of a files place holds it under a fresh token; `save`
  refuses an older handle and `release` on one does nothing, so a
  superseded attempt cannot flush or unlink its replacement's files.
- A run whose phase is done but whose final save failed fails the
  attempt instead of completing, so the retry saves the files.
- A relative fixed `sourceRepo` is resolved before it reaches the host,
  which takes only absolute paths as local repositories.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KTy6FK6trgHQhStU5yxgi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants