Skip to content

fix(core): resourceTools reads only llmReadable collections, without secret fields (FIX-2436) - #4426

Merged
jhoffner merged 6 commits into
mainfrom
claude/project-thread-5l6qv4
Nov 1, 2026
Merged

jhoffner merged 6 commits into
mainfrom
claude/project-thread-5l6qv4

Conversation

@jhoffner

@jhoffner jhoffner commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator

Requested by Jake · project thread

What

FIX-2436 (bug, child of epic FIX-2369, blocks closure FIX-2425). Direct route, no spec.

Core's generic resourceTools() readResource({ path }) resolved a path against every registered collection and returned the full stored state. A worker given resourceTools() beside a collection it may not show the model, such as a user's MCP keys (mcp-credentials/*, which #4415 installs on a worker's model block), could read the key.

Now readResource:

  • reaches only a collection that opted in with llmReadable, the same gate the content and search tools already apply. Any other path answers as one no collection matches, so it doesn't reveal that the collection exists (the convention resourceVisibility uses for hidden resources);
  • returns state without the collection's secret fields (omitSecretFields, the helper every engine read already uses).

Generic L1 rule: it honours the collection's own flags, and no Workforce noun enters core.

Worth your eye

  • Behaviour change in a published package. A consumer that used readResource on a collection without llmReadable now gets "No resource collection found". No app in the repo uses resourceTools(). Changeset added (@flow-state-dev/core patch) saying to mark the collection llmReadable: true.
  • Writes are unchanged. create/update/deleteResource still reach any collection the visibility rule leaves writable; they return only { ok }, so they can't leak a key, but a model could overwrite or delete one. Gating state writes on llmWritable would widen that flag's meaning (today it covers content writes). I left it out of this bug; say if you want it filed.
  • The test uses mcpCredentialsCapability, not projectMcpCapability. feat(workforce): workers get their project's MCP servers as tools (FIX-2388 PR 2) #4415 is still open, so projectMcpCapability isn't on main. The test holds the same collection object projectMcpCapability installs, on an agent worker's model, with readResource picked from its catalog, and the model calls readResource({ path: "mcp-credentials/linear" }). Once feat(workforce): workers get their project's MCP servers as tools (FIX-2388 PR 2) #4415 merges, swapping the capability is a one-line change if you want it.

Verification

unset FSDEV_DEFAULT_MODEL before every run.

  • packages/workforce/test/resource-tools-key-read.test.ts (new): one agent worker holds the keys collection and readResource; a user's key is stored; the model asks for it. The tool answers "No resource collection found matching path: mcp-credentials/linear" and the key appears nowhere in the run's items.
  • packages/engine/test/resource-tools-path-matching.test.ts: fixture collections marked llmReadable so every existing row keeps its meaning; two rows added: a collection that didn't opt in answers as unmatched, and an opted-in collection is read without its secret field.
  • Suites: core 164 files / 2613 tests pass. Engine 315/317 files pass; the two failures (get-items-hot-path, store-factory-defaults) are timing tests that pass in isolation on this branch. Workforce 147/148 files pass; coordinator-rounds times out on main too.
  • pnpm --filter @flow-state-dev/core typecheck: same 155 errors as main in this checkout (unbuilt workspace), none in the changed files. eslint clean; pre-push vocabulary sweep clean.

Negative controls, each applied to the source and re-run (each goes red):

Control Failing test
drop the llmReadable check the worker test (the read succeeds) · read vault/linear row
drop both checks (main's code) the worker test: the tool output carries "value":"lin_api_users_own_key"
drop omitSecretFields read keys/linear row

Docs

packages/core/README.md (Resource CRUD tools paragraph) and the llmReadable doc comment on ResourceCollectionConfig. No apps/docs page mentions resourceTools().


Generated by Claude Code

…secret fields (FIX-2436)

readResource resolved a path against every registered collection and
returned the full stored state, so a worker given resourceTools() beside a
collection it may not show the model (a user's MCP keys) could read it.
The read now reaches only a collection that opted in with llmReadable, the
gate the content and search tools already apply, and drops the fields the
collection names in secret. Any other path answers as one no collection
matches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NczDiGRHaqqMGz8cJxCeoQ
@jhoffner jhoffner self-assigned this Nov 1, 2026
@vercel

vercel Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
flow-state-dev-build-site Ready Ready Preview Nov 1, 2026 12:50am UTC
3 Skipped Deployments
Project Deployment Actions Updated
flow-state-dev-docs Ignored Ignored Nov 1, 2026 12:50am UTC
flow-state-dev-kitchen-sink Ignored Ignored Nov 1, 2026 12:50am UTC
flow-state-dev-ui Ignored Ignored Preview Nov 1, 2026 12:50am UTC

Request Review

@linear-code

linear-code Bot commented Nov 1, 2026

Copy link
Copy Markdown

FIX-2436

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-01T00:14:53.322406Z d0e148c New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

jhoffner commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author

Second look + architecture pass (FIX-2436)

Verdict: the approach is right and about as lean as it gets. Reusing the collection's own llmReadable and secret flags avoids a new concept, and no Workforce noun enters core. Nothing blocking on the read path. Two things are worth the implementer's attention, one small and one a real follow-up.

1. Real gap: writes still reach the key (follow-up, not necessarily this PR)

create/update/deleteResource still reach any collection with access === "visible", with no llmWritable check. Concrete consequence for the motivating case: a prompt-injected worker holding resourceTools() beside mcp-credentials/* can't read the key but can deleteResource it, or updateResource it to an attacker-chosen value, so later MCP calls go out under the wrong credential. The PR description notes this and defers it. I agree it shouldn't ride in this bug fix, but I'd file the issue before FIX-2425 closes, not "if asked". Otherwise the epic closes with the read half of a symmetric hole fixed.

On the PR's worry that gating on llmWritable widens that flag's meaning: the content tools already gate writes on isLlmWritable ("Independent of llmReadable"), and isLlmWritableNow exists for the visibility-rule case. Applying the same gate to state writes makes the flag mean "the model may write this collection through any generic tool", which is the more coherent reading and is what a reader would assume. Recommend that shape for the follow-up. It is the same one-line filter this PR adds on the read side. It is a behaviour change, so it needs its own changeset.

2. Small reuse nit: the gate is now written inline a fourth time

ns.ref.config?.llmReadable === true now appears in resolvePathToCollection, collectReadableResources, collectReadableProjectedCollections, and isLlmReadable. The doc comment on collectReadableProjectedCollections says it directly: "a gate written twice is a gate that can drift so one of them leaks." isLlmReadable(ref) is exported in the same file. Use isLlmReadable(ns.ref) in the new filter (resource-tools.ts ~L364) so there is one expression. Don't refactor the other two callers in this PR.

3. Nits

  • resource-collection.ts doc comment: the reflowed llmReadable paragraph now has one over-long line (...which also leaves out secret fields. Collection-wide — ...). Rewrap it.
  • packages/workforce/test/resource-tools-key-read.test.ts is the right end-to-end repro. It overlaps with the two engine rows on the core behaviour, which is acceptable, since the engine rows pin the rule and this one pins the scenario. Once feat(workforce): workers get their project's MCP servers as tools (FIX-2388 PR 2) #4415 lands, swapping in projectMcpCapability is worth doing so the test tracks the real installer rather than a stand-in.
  • omitSecretFields strips top-level fields only. The README says "a field the collection names in secret", which is accurate. No change needed, just confirming the nested-secret case isn't claimed.

Architecture (improve-codebase-architecture lens)

No refactor is justified by this PR. The one structural smell is the repeated read-gate expression above, which the one-line reuse fixes without a new abstraction. A broader "single visibility gate for all generic resource tools" module would be speculative until the write-gate follow-up shows what shape both sides share, so revisit it there.


Generated by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 40814a1200

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .changeset/resource-tools-read-gate.md Resolved
…k a consumer (FIX-2436)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NczDiGRHaqqMGz8cJxCeoQ

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/simplify review (FIX-2436)

Verdict: approve. This is a small, behavior-aligned fix, not an over-built layer. The risky alternative would have been a new abstraction or a second resolution path; instead you extended the existing resolvePathToCollection(..., intent) split (read vs write) and reused omitSecretFields, which is the same primitive client-projection already documents for server-leaving reads.

Did the approach go too far?

No — if anything it stayed appropriately narrow:

Alternative Why it would be more complex or worse
Gate only inside readResource after get() (content-tool style: resolve, then isLlmReadable(ref)) Still need matching logic; may fetch secret state before rejecting; error shape may diverge from "no collection matches" unless you rethrow carefully.
Route reads through resolveClientData / full client projection Changes what agents see for collections with client.expose / data; not equivalent to "raw state minus secret".
New package or wrapper around CRUD tools Cross-import smell for ~15 lines of policy already owned by resource-tools.ts.
Fold readResource into content/search tools only Different contract (path vs uri, state vs rendered content); FIX-2436 is specifically generic CRUD beside workforce keys.

Putting the llmReadable filter on read intent in resolvePathToCollection matches how search/discovery skip non-readable collections before touching instances (lazy collections, no accidental bulk work). That is the cleaner symmetry, not over-engineering.

Simplify follow-ups (non-blocking)

  1. Duplicate predicate — The module already warns that a gate written twice can drift (collectReadableProjectedCollections). This PR adds a fourth copy of ref.config?.llmReadable === true (also in collectReadableResources). Consider a single private helper (or widening isLlmReadable to accept anything with { config?: { llmReadable?: boolean } }) and using it in all four places. Same behavior, less future drift.

  2. Tool description — Block comment and README mention opt-in + secrets; readResource's generator-facing description still reads like an unconstrained read. Aligning one clause there would reduce surprise (product/docs nit).

  3. Workforce integration test — resource-tools-key-read.test.ts is a fair regression for real mcpCredentialsCapability wiring, but the engine table already covers non-llmReadable → NO_COLLECTION and secret stripping. If CI time or maintenance cost matters, you could drop the workforce file and rely on engine coverage; keeping it is also defensible for the FIX-2436 story.

  4. Out of scope (mention only) — Write CRUD still resolves on visibility, not llmWritable; content tools use isLlmWritableNow. I am not asking to expand this PR, but if someone expects "non-llmReadable collections are inert for all generic tools," writes may still surprise them. Worth a separate issue if that is a product requirement.

Checks

Ran pnpm test resource-tools-path-matching in @flow-state-dev/engine — 56 tests passed on this branch.

Performance: no meaningful concerns (filter may reduce work on read; omitSecretFields is a no-op copy when secret is empty).

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

…ource (FIX-2436)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NczDiGRHaqqMGz8cJxCeoQ

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Snob. A smaller shape exists. The read contract stays the same.

isCollectionLlmReadable is isLlmReadable on ns.ref. Widen isLlmReadable so it takes a resource ref or a collection ref, and use it for listing, projected search, and readResource. Delete the helper.

POC, do not merge: #4428

Fold that subtraction into this PR. After this PR lands, close the POC unmerged.

Open in Web View Automation 

Sent by Cursor Automation: Code Snob

…d gate (FIX-2436)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NczDiGRHaqqMGz8cJxCeoQ

jhoffner commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author

Kitchen-sink E2E (Playwright) failed on d0e148c, and the cause isn't this PR. The job never got to run tests: ubuntu-latest now resolves to ubuntu-26.04, and Playwright refuses to install chromium-headless-shell there ("Playwright does not support chromium-headless-shell on ubuntu26.04-x64"). FIX-2438 (#4431) fixes this by pinning the job to ubuntu-24.04. I ported that same one-line change into this PR in 9c9c7d1, so CI here can go green before #4431 merges. Once both are on main they merge cleanly, since the change is identical.


Generated by Claude Code

@jhoffner
jhoffner merged commit 824ce87 into main Nov 1, 2026
20 checks passed
@jhoffner
jhoffner deleted the claude/project-thread-5l6qv4 branch November 1, 2026 01:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants