Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/local-checkout-starts-from-upstream.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"@flow-state-dev/workspace": minor
"@flow-state-dev/harness-manager": patch
---

A repository listed in `localRepositories` now fetches the base's upstream before it cuts a new run's branch, and cuts from whichever of local and upstream is newer. A base whose local and upstream copies have each moved is refused as `base-diverged`; a failed fetch falls back to the last-fetched copy with a warning. `place.repo.baseCommit` is now set for these repositories, and `readRunCommits` also leaves out the base's upstream (FIX-1927).
8 changes: 6 additions & 2 deletions apps/docs/docs/shift-manager/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,13 @@ If the Lab fails to start after a save, the error prints and Shift Manager waits

In this repository, `pnpm shift-manager:dev` opens the `fsd` profile: a Lab for working on this repository with Shift Manager itself. It has two projects from the start. **FSD** is the framework packages, and **Shift Manager** is this app. Both are shared with everyone in the Lab, and both point at this repository. Inbox is empty until a worker asks you something.

Ask the [chief of staff](../workforce/chief-of-staff.md) for work on either project, and say which one, for example "on Shift Manager, make the empty Inbox say who to ask". A coding agent picks it up in a fresh worktree of your checkout, on a branch of its own cut from your local `main`. It doesn't fetch first, so pull `main` when you want the run to start from the latest. Review the work in that worktree, or from your checkout with `git diff main...<branch>`. The branch stays checked out in its worktree, so git won't check it out a second time in yours.
Ask the [chief of staff](../workforce/chief-of-staff.md) for work on either project, and say which one, for example "on Shift Manager, make the empty Inbox say who to ask". A coding agent picks it up in a fresh worktree of your checkout, on a branch of its own. Before cutting that branch, the Lab fetches `main` from GitHub and starts from whichever is newer: that, or your local `main`. So you don't need to pull first, and commits you haven't pushed are kept. Your own `main`, its files and your index aren't touched.

Only those two projects get a worktree of your checkout. The Lab reaches no git remote, so a coding run on a project you add yourself, even one on this repository, is refused, naming the repository, and nothing is fetched.
If your `main` and `origin/main` each have commits the other lacks, the run is refused before anything is made, and the refusal names both commits. Pull or rebase, or push, then ask again. If the fetch fails, because you're offline for instance, the run starts from the `origin/main` you last fetched, and a warning prints where you started Shift Manager.

Review the work in that worktree, or from your checkout with `git log -p <branch> --not main origin/main`. That shows only the run's commits, whichever of the two it started from. The branch stays checked out in its worktree, so git won't check it out a second time in yours.

Only those two projects get a worktree of your checkout. Apart from that fetch, the Lab reaches no git remote, so a coding run on a project you add yourself, even one on this repository, is refused, naming the repository, and nothing is cloned.

The coding agent is Claude Code. Sign in to Claude Code, or set `ANTHROPIC_API_KEY`, before you start. Without either, the Lab refuses to start and says so. To run the scripted coder (`stub`) instead, which writes one file and costs nothing, set `DEVFORCE_LAB_HARNESS=stub`. A `Coder:` line at start names the coder it started with. The chief of staff answers on a model of its own, so its turns need `ANTHROPIC_API_KEY` or `AI_GATEWAY_API_KEY`.

Expand Down
15 changes: 13 additions & 2 deletions apps/docs/docs/workforce/projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -543,11 +543,22 @@ Only shared projects can be listed, each by its organization and id. A run uses

The checkout also has to be in the host's `localRepositories`. That list is what lets the host touch a folder on its machine at all. If the two disagree, the run is refused with `invalid-remote`, naming the path, before git runs. A relative `checkout` path, a `remote` a project couldn't store, or a project listed twice is refused when `projectWorkspace()` is called, naming the entry.

Each run's place has the worktree in `checkout/` and the [project's files](#the-projects-own-files) in `project/` beside it, as for any other repository. The branch is cut from `baseRef`, or from whatever your checkout has checked out when `baseRef` is left out. Nothing is fetched, so `main` is whatever you last pulled. The server never switches, resets or cleans your own working tree.
Each run's place has the worktree in `checkout/` and the [project's files](#the-projects-own-files) in `project/` beside it, as for any other repository. The branch is cut from `baseRef`, or from whatever your checkout has checked out when `baseRef` is left out.

Before the server cuts a new branch, it fetches that base's upstream, the remote branch it tracks, such as `origin/main`. Then it starts from whichever is newer: upstream when you haven't pulled, your own branch when it has commits you haven't pushed. Only the remote-tracking copy is updated. The server never switches, resets or cleans your working tree, and never moves your branch.

A few cases go differently:

- **Your branch and its upstream have each moved.** The run is refused with `base-diverged`, naming both commits, and nothing is made. Pull or rebase, or push, and the next run starts.
- **The fetch fails**, because you're offline, a credential is refused, or it times out. The run starts from the copy you last fetched, and the server logs a warning naming the checkout and git's error.
- **The base has no upstream**, such as a tag, a commit, or a branch that tracks nothing. It is used as it is, with no fetch.
- **The run already has its branch**, on a retry or a later turn. It's handed back as it was, and nothing is fetched.

Listing the checkout is what allows the fetch, so the upstream doesn't need to be in `remotes.allow`. That list still decides which remotes the host clones. The upstream's URL is checked before git sees it: one that uses a transport helper such as `ext::`, starts with `-`, or uses a scheme other than `https`, `ssh` or `file` isn't fetched, and the server logs why.

Several projects can share one checkout, and their runs work side by side, each in its own worktree on its own branch.

To review a run, open its worktree, or run `git diff main...<branch>` from your checkout. The branch stays checked out in the run's worktree, because a run can be continued later, after a question is answered or when you send it another turn, and it picks up in the same place. So git won't check that branch out a second time in your own tree. When you're done with a run, `git worktree remove <path>` frees the branch.
To review a run, open its worktree, or run `git log -p <branch> --not main origin/main` from your checkout. That shows only the run's commits, whether it started from your `main` or from `origin/main`. `git diff main...<branch>` would also show upstream commits your `main` doesn't have yet. The branch stays checked out in the run's worktree, because a run can be continued later, after a question is answered or when you send it another turn, and it picks up in the same place. So git won't check that branch out a second time in your own tree. When you're done with a run, `git worktree remove <path>` frees the branch.
Comment thread
jhoffner marked this conversation as resolved.
Resolved

A fresh worktree has no `node_modules`. The coding agent installs them the way your repository tells it to. With pnpm that comes from the machine's shared store and takes seconds once the checkout has been installed.

Expand Down
15 changes: 9 additions & 6 deletions packages/harness-manager/src/commits.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,17 +28,20 @@ async function git(cwd: string, args: string[]): Promise<string> {
}

/**
* The commits on `cwd`'s HEAD that `baseRef` does not have, in either copy:
* the remote's (`origin/<baseRef>`) or the local branch. A local checkout
* cuts the run's branch from the local branch and a clone from the remote's,
* and either copy can be ahead of the other, so a commit either one has is
* not the run's. No base, or no checkout git can read, answers none.
* The commits on `cwd`'s HEAD that `baseRef` does not have, in any copy: the
* remote's (`origin/<baseRef>`), the local branch, or the local branch's
* upstream (`<baseRef>@{upstream}`, whatever its remote is called). A local
* checkout cuts the run's branch from the newer of the local branch and its
* upstream, and a clone from the remote's, and any copy can be ahead of
* another, so a commit any one has is not the run's. No base, or no checkout
* git can read, answers none.
*/
export async function readRunCommits(cwd: string, baseRef: string | null): Promise<RunCommits> {
if (baseRef === null) return { count: 0, subjects: [] };
const bases: string[] = [];
// In the run's own checkout, a bare `HEAD` is the run's branch, never its base.
const copies = baseRef === "HEAD" ? [`refs/remotes/origin/HEAD`] : [`refs/remotes/origin/${baseRef}`, baseRef];
const copies =
baseRef === "HEAD" ? [`refs/remotes/origin/HEAD`] : [`refs/remotes/origin/${baseRef}`, baseRef, `${baseRef}@{upstream}`];
Comment thread
jhoffner marked this conversation as resolved.
Resolved
Comment thread
jhoffner marked this conversation as resolved.
Resolved
for (const base of copies) {
try {
await git(cwd, ["rev-parse", "--verify", "--quiet", `${base}^{commit}`]);
Expand Down
5 changes: 3 additions & 2 deletions packages/harness-manager/src/workspace.ts
Original file line number Diff line number Diff line change
Expand Up @@ -552,8 +552,9 @@ export const ASK_MARKER_IGNORED: IgnoredDirectory = {
/**
* The workspace host a fixed repository runs through: `sourceRepo` is a
* repository on this machine the operator named, so the host cuts worktrees
* from it directly, keeps every run's branch in it, and reaches no remote.
* The run source is constant.
* from it directly and keeps every run's branch in it. It reaches only the
* base's upstream, to cut a new branch from the newer of the two. The run
* source is constant.
*/
export function hostForConfig(config: WorkspaceConfig, now: () => number = Date.now): WorkspaceHost {
// Absolute, because the host takes only an absolute path as a repository on
Expand Down
24 changes: 24 additions & 0 deletions packages/harness-manager/test/commits.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,30 @@ describe("readRunCommits", () => {
expect(await readRunCommits(dir, "main")).toEqual({ count: 1, subjects: ["the run's change"] });
});

it("counts none of the upstream's commits when the branch was cut from it, whatever its remote is called", async () => {
// A local checkout cuts from the base's upstream when the person hasn't
// pulled. Named anything but `origin`, only `<base>@{upstream}` finds it.
const dir = repo("local");
git(dir, "update-ref", "-d", "refs/remotes/origin/main");
git(dir, "config", "remote.upstream.url", dir);
git(dir, "config", "remote.upstream.fetch", "+refs/heads/*:refs/remotes/upstream/*");
git(dir, "config", "branch.main.remote", "upstream");
git(dir, "config", "branch.main.merge", "refs/heads/main");
git(dir, "checkout", "-q", "-b", "tmp", "main~1");
commit(dir, "upstream only");
git(dir, "update-ref", "refs/remotes/upstream/main", "HEAD");
git(dir, "checkout", "-q", "main");
git(dir, "reset", "-q", "--hard", "main~1");
git(dir, "branch", "-q", "-D", "tmp");
git(dir, "checkout", "-q", "-b", "run", "upstream/main");
commit(dir, "the run's first change");
commit(dir, "the run's second change");
expect(await readRunCommits(dir, "main")).toEqual({
count: 2,
subjects: ["the run's second change", "the run's first change"],
});
});

it("answers none for a base the checkout doesn't have", async () => {
const dir = repo("local");
expect(await readRunCommits(dir, "no-such-branch")).toEqual({ count: 0, subjects: [] });
Expand Down
24 changes: 24 additions & 0 deletions packages/harness-manager/test/workspace.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,30 @@ describe("provisioning", () => {
expect(readFileSync(join(first.path, "wip.txt"), "utf8")).toBe("half done");
});

it("starts a new run from the source repository's upstream when it is ahead of main", async () => {
// The fixed `sourceRepo` goes through the same host path as the Lab: a
// person who hasn't pulled still gets a run cut from the current base.
const config = workspace();
const git = (cwd: string, ...args: string[]) =>
execFileSync("git", ["-c", "user.name=t", "-c", "user.email=t@t", ...args], { cwd, encoding: "utf8" }).trim();
const upstream = join(config.root, "..", "upstream.git");
git(config.sourceRepo, "clone", "-q", "--bare", config.sourceRepo, upstream);
git(config.sourceRepo, "remote", "set-url", "origin", upstream);
git(config.sourceRepo, "fetch", "-q", "origin");
git(config.sourceRepo, "branch", "-q", "--set-upstream-to=origin/main", "main");
const teammate = join(config.root, "..", "teammate");
git(join(config.root, ".."), "clone", "-q", upstream, teammate);
git(teammate, "commit", "-q", "--allow-empty", "-m", "merged upstream");
git(teammate, "push", "-q", "origin", "main");
const merged = git(teammate, "rev-parse", "HEAD");
const localMain = git(config.sourceRepo, "rev-parse", "main");

const checkout = await provisionCheckout(config, at("FIX-1219", "implement"));

expect(git(checkout.path, "rev-parse", "HEAD")).toBe(merged);
expect(git(config.sourceRepo, "rev-parse", "main")).toBe(localMain);
});

it("puts the checkout on its own branch", async () => {
const config = workspace();
const checkout = await provisionCheckout(config, at("FIX-1219", "implement"));
Expand Down
8 changes: 6 additions & 2 deletions packages/shift-manager/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,13 @@ If the Lab fails to start after a save, the error prints and Shift Manager waits

In this repository, `pnpm shift-manager:dev` opens the `fsd` profile, at `teams/devteam/fsd.fsdev.config.mts`: a Lab for working on this repository with Shift Manager itself. It has two projects from the start. **FSD** is the framework packages, and **Shift Manager** is this app. Both are shared with everyone in the Lab, and both point at this repository. It raises no ask and opens no workstream, so Inbox is empty until a worker asks you something.

Ask the chief of staff for work on either project, and say which one, for example "on Shift Manager, make the empty Inbox say who to ask". A coding agent picks it up in a fresh worktree of your checkout, on a branch of its own cut from your local `main`. It doesn't fetch first, so pull `main` when you want the run to start from the latest. Review the work in that worktree, or from your checkout with `git diff main...<branch>`. The branch stays checked out in its worktree, so git won't check it out a second time in yours.
Ask the chief of staff for work on either project, and say which one, for example "on Shift Manager, make the empty Inbox say who to ask". A coding agent picks it up in a fresh worktree of your checkout, on a branch of its own. Before cutting that branch, the Lab fetches `main` from GitHub and starts from whichever is newer: that, or your local `main`. So you don't need to pull first, and commits you haven't pushed are kept. Your own `main`, its files and your index aren't touched.

Only those two projects get a worktree of your checkout. The Lab reaches no git remote, so a coding run on a project you add yourself, even one on this repository, is refused, naming the repository, and nothing is fetched.
If your `main` and `origin/main` each have commits the other lacks, the run is refused before anything is made, and the refusal names both commits. Pull or rebase, or push, then ask again. If the fetch fails, because you're offline for instance, the run starts from the `origin/main` you last fetched, and a warning prints where you started Shift Manager.

Review the work in that worktree, or from your checkout with `git log -p <branch> --not main origin/main`. That shows only the run's commits, whichever of the two it started from. The branch stays checked out in its worktree, so git won't check it out a second time in yours.

Only those two projects get a worktree of your checkout. Apart from that fetch, the Lab reaches no git remote, so a coding run on a project you add yourself, even one on this repository, is refused, naming the repository, and nothing is cloned.

The coding agent is Claude Code. Sign in to Claude Code, or set `ANTHROPIC_API_KEY`, before you start. Without either, the Lab refuses to start and says so. To run the scripted coder (`stub`) instead, which writes one file and costs nothing, set `DEVFORCE_LAB_HARNESS=stub`. A `Coder:` line at start names the coder it started with. The chief of staff answers on a model of its own, so its turns need `ANTHROPIC_API_KEY` or `AI_GATEWAY_API_KEY`.

Expand Down
3 changes: 2 additions & 1 deletion packages/shift-manager/teams/devteam/fsd-checkouts.mts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ const CHECKOUT = resolve(fileURLToPath(new URL("../../../..", import.meta.url)))

/**
* Where the two projects' runs work: worktrees of {@link CHECKOUT}, each on a
* branch cut from local `main`, never fetched. Only the two shared projects
* branch cut from the newer of local `main` and its freshly fetched upstream
* (the workspace host's rule for a listed checkout). Only the two shared projects
* of the Lab's organization are named, so a project a user makes on the same
* remote is not mapped, and is refused.
*/
Expand Down
10 changes: 6 additions & 4 deletions packages/shift-manager/teams/devteam/fsd.fsdev.config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,13 @@
* what to set. `DEVFORCE_LAB_HARNESS=stub` runs the scripted coder instead.
* - **Runs work in this checkout.** A coding run on FSD or Shift Manager gets
* a fresh worktree of the checkout this file sits in, on a branch of its
* own cut from local `main` ({@link CHECKOUTS}, in `fsd-checkouts.mts`). The
* checkout's location is worked out from that file's, never written down.
* - **No remote is reached.** The workspace hosts allow none, so a run the
* own cut from the newer of local `main` and its upstream ({@link CHECKOUTS},
* in `fsd-checkouts.mts`). The checkout's location is worked out from that
* file's, never written down.
* - **No remote is cloned.** The workspace hosts allow none, so a run the
* mapping misses (a project a user made, or one in another organization) is
* refused, naming the remote, and nothing is fetched.
* refused, naming the remote, and nothing is cloned. The one remote reached
* is the checkout's own upstream, fetched before a new run's branch is cut.
* - **Its own store**: SQLite at `FSD_STORE` or
* `packages/shift-manager/.fsdev/fsd.sqlite`. DevTeam's rows never show
* here. Delete the file for a fresh start.
Expand Down
Loading
Loading