Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/resource-secret-fields.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@flow-state-dev/core": patch
"@flow-state-dev/engine": patch
"@flow-state-dev/workforce": patch
---

`defineResource` and `defineResourceCollection` take a `secret` list of state fields the engine never sends to a client or returns from the debug routes, and Workforce marks a stored MCP key's `value` secret so the debug routes no longer show it (FIX-1955).
2 changes: 2 additions & 0 deletions apps/docs/docs/api/core.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,7 @@ Resource options:
- `render?: (content, state) => string | Promise<string>` — optional renderer for `readContent()`
- `llmReadable?: boolean` — allows read access when `readResourceContentTool()` is installed
- `llmWritable?: boolean` — allows write access when `writeResourceContentTool()` is installed
- `secret?: Array<keyof State>` — state fields left out of client snapshots, state endpoints, `live` deltas and the debug endpoints (content is not covered). See [Secret fields](/docs/resources/client-access#secret-fields)

Runtime resource state methods (`ResourceRef`). The mutators resolve to `void` — read the result back off the synchronous `ref.state` getter:

Expand Down Expand Up @@ -257,6 +258,7 @@ Config options:
- `llmReadable?: boolean` — exposes every instance's content to `readResourceContentTool()` and content search (`grepResourceContent` / `searchResources`). Default `false`
- `llmWritable?: boolean` — lets `writeResourceContentTool()` overwrite an instance body. Default `false`; independent of `llmReadable` and of `writable`
- `stampOrgId?: boolean` — when `true`, creating an instance whose initial state has no `orgId` writes the organization the run belongs to into it. Creating an instance with an `orgId` other than the run's organization throws. On any write to an existing instance (including `upsert` and `create(..., { replace: true })`), the stored `orgId` stands: leaving it out keeps it, and naming a different one throws without writing. An instance with no stored `orgId` keeps none unless the write names the run's own organization; naming any other throws. The `stateSchema` must declare `orgId`, or `defineResourceCollection` throws. Default `false`
- `secret?: Array<keyof State>` — state fields left out of client snapshots, state endpoints, `live` deltas and the debug endpoints (content is not covered). See [Secret fields](/docs/resources/client-access#secret-fields)
- `onInstanceCreated?: (key, state, ctx) => void` — lifecycle hook
- `onInstanceUpdated?: (key, state, prevState, ctx) => void` — lifecycle hook
- `onInstanceDeleted?: (key, ctx) => void` — lifecycle hook
Expand Down
8 changes: 4 additions & 4 deletions apps/docs/docs/devtool/debug-vs-client-state.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ There are two ways to look at a session.

Production clients (your React app, your CLI, anything calling the public API) read the session through the normal endpoints. What they get back is filtered: each resource's `client` config decides which fields ship, and collections obey `prefetchWindow` so a client only sees the items it asked for.

The DevTool talks to a different surface. It calls a privileged debug endpoint that returns the raw server-side state, ignoring `client.data` projections and `prefetchWindow`, with one roster exception [covered below](#what-the-debug-endpoint-returns). It's the same session, but the DevTool sees it the way the runtime sees it.
The DevTool talks to a different surface. It calls a privileged debug endpoint that returns the raw server-side state, ignoring `client.data` projections and `prefetchWindow`, except for one roster case and any secret fields, both [covered below](#what-the-debug-endpoint-returns). It's the same session, but the DevTool sees it the way the runtime sees it.

## What the debug endpoint returns

For each storage key the session touches, the debug response includes:

- The full state body (every field on the state schema)
- The full state body (every field on the state schema except those named in [`secret`](../resources/client-access.md#secret-fields))
- Content metadata
- The underlying resource definition, with its alias list
- A second copy showing the projection your `client.data` would produce
- A second copy showing the projection your `client.data` would produce, also without secret fields
- Whether the resource may be written, and whether a model is offered a write tool for its content. Either can be absent; [Read-only resources](#read-only-resources) covers what that means

It does not apply `prefetchWindow`: a collection lists all of its items, not a window of them. It does not apply `client.data` to the storage view either; that projection is shown alongside the raw state, not in place of it.
Expand Down Expand Up @@ -135,7 +135,7 @@ A request with no `Origin` header, such as curl, a script, or a same-origin GET

## Don't ship it enabled to production

The endpoint is read-only, but it exposes the full server-side state of every session, including fields your `client.data` deliberately hides. That's the point of it as a debugging tool, and it's also the reason you don't want it reachable from the public internet.
The endpoint is read-only, but it exposes the full server-side state of every session, including fields your `client.data` hides. That's the point of it as a debugging tool, and it's also the reason you don't want it reachable from the public internet. It never returns fields named in [`secret`](../resources/client-access.md#secret-fields).

If you have a legitimate reason to enable it in a non-local environment (a staging tier where the DevTool runs on a known internal origin), audit `debugAllowedOrigins` carefully and put the route behind whatever authentication your platform already enforces.

Expand Down
2 changes: 1 addition & 1 deletion apps/docs/docs/devtool/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ A read that fails is never shown as an empty section. A refused read names its s

Inspect current state at every scope level. View session-level state, user-level state, and org-level state. Resources and their content are visible. ClientData values appear in the detail panel.

The Resources panel reads from a privileged debug endpoint and shows the full server-side state, except for workforce roster workers owned by a user other than the session's user. Each entry can be toggled between the raw server view, the client view (what production clients receive after `client.data` projection), and a diff between the two. Production clients see only what each resource's `client` config allows. See [Debug vs client state](./debug-vs-client-state.md) for the mental model and how to enable the endpoint locally.
The Resources panel reads from a privileged debug endpoint and shows the full server-side state, except for workforce roster workers owned by a user other than the session's user and any state field a resource names in [`secret`](../resources/client-access.md#secret-fields). Each entry can be toggled between the raw server view, the client view (what production clients receive after `client.data` projection), and a diff between the two. Production clients see only what each resource's `client` config allows. See [Debug vs client state](./debug-vs-client-state.md) for the mental model and how to enable the endpoint locally.

## Replay

Expand Down
33 changes: 32 additions & 1 deletion apps/docs/docs/resources/client-access.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,37 @@ Pick one. If you need both whitelisting and computed fields, use `data` and writ

Reach for `expose` first; use `exclude` when you have many fields and only need to hide a few; reserve `data` for computed fields that aren't on the state schema.

## Secret fields

A projection decides what the client sees. It doesn't cover the [debug endpoints](/docs/devtool/debug-vs-client-state) the DevTool reads, which return a resource's full stored state. For a field that must never leave the server, such as a stored API key, name it in `secret`. It's a top-level option on `defineResource()` and `defineResourceCollection()`, not part of `client`:

```ts
const apiKeys = defineResourceCollection({
pattern: "api-keys/*",
scope: "user",
stateSchema: z.object({ value: z.string(), setAt: z.string() }),
secret: ["value"],
client: { state: { read: true }, expose: ["setAt"] },
});
```

A secret field is left out of every state read the server answers: the session snapshot's `clientData` (prefetched collection items included), the collection list and item-state endpoints, `live` deltas, and the debug endpoints. A `client.data` function sees the full state, so it can derive a value from a secret field (`hasKey: s.value !== ""`), but a secret field in what it returns is dropped. Your blocks read and write the field as usual through the resource handle.

A name that isn't on the state schema (here `secret: ["valu"]`), or one that `client.expose` also lists, throws at definition time:

```
defineResourceCollection() for "api-keys/*": secret names key(s) not on stateSchema: valu. Valid keys: setAt, value.
defineResourceCollection() for "api-keys/*": client.expose names secret key(s): value. A secret field never leaves the server; remove it from one list.
```

A user or org collection that several flows share is one stored row per key, so every flow that declares it must name the same secret fields. Registering a flow whose declaration disagrees throws.

What `secret` doesn't do:

- It covers state fields, not content. A `contentTemplate` that interpolates a secret field renders its value into the content body, and content reads return it.
- It doesn't filter a scope's `client.derived` values. Those functions read resource state in full, the same as `client.data`, and whatever they return is sent under the names they give it. Derive from a secret field there (`hasKey`), don't return it.
- It doesn't change the derived client type ([`ClientDataOf`](#typing-clientdata)) under the identity or `exclude` projection, which still lists the secret field. Pair `secret` with `expose`, as above, so the type matches what arrives.

## Collection example

Collections are where client access gets the most use. A typical artifact or file collection exposes metadata for listing and content for viewing:
Expand Down Expand Up @@ -592,7 +623,7 @@ The nullable fields follow the resource-schema default convention (`.nullable().

## Debug and the DevTool

During development, the DevTool's Resources panel reads from a privileged debug endpoint that ignores everything on this page. It shows you the full server-side state, including fields and items that `client.data` deliberately drops. That's the point of an inspector: the DevTool is talking to the runtime directly, not pretending to be a client.
During development, the DevTool's Resources panel reads from a privileged debug endpoint that ignores everything on this page. It shows you the full server-side state, including fields and items that `client.data` drops. That's the point of an inspector: the DevTool is talking to the runtime directly, not pretending to be a client. It never returns fields named in [`secret`](#secret-fields).

The panel renders each resource with a Raw / Client / Diff toggle. **Raw** is the unfiltered server state. **Client** runs the same `expose`, `exclude`, or `data` projection your production app would receive. **Diff** shows both side by side so you can see which fields survive and which get stripped.

Expand Down
80 changes: 79 additions & 1 deletion packages/core/src/helpers/client-projection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@
* 1. Mutual exclusion across `expose`, `exclude`, and `data`.
* 2. Field names listed in `expose` / `exclude` exist on the state schema
* (best-effort — silently skipped when the schema isn't a `ZodObject`).
* 3. `secret` fields exist on the state schema and are not in `expose`.
*
* `omitSecretFields` strips a config's `secret` fields from raw state, for the
* debug reads that return it. `resolveClientData` is the client view every
* other engine read sends: the projection run on the full state, with any
* `secret` field dropped from what it returns.
*
* `resolveClientProjection` runs at request time (server routes) and produces
* the `clientData` payload from a resource/collection-item state. The four
Expand Down Expand Up @@ -65,8 +71,10 @@ export function validateClientProjection(args: {
kind: "single" | "collection";
stateSchema: unknown;
client: ProjectionClient;
secret?: ReadonlyArray<string>;
}): void {
const { definer, ref, kind, stateSchema, client } = args;
const { definer, ref, kind, stateSchema, client, secret } = args;
if (secret !== undefined) validateSecretFields(definer, ref, stateSchema, client, secret);
if (client === undefined) return;

const set: string[] = [];
Expand Down Expand Up @@ -115,6 +123,76 @@ export function validateClientProjection(args: {
);
}

/**
* A `secret` field must be on the state schema (best-effort, as for `expose`)
* and must not be one `client.expose` asks to send: the two contradict, and
* which wins would be a guess.
*/
function validateSecretFields(
definer: string,
ref: string,
stateSchema: unknown,
client: ProjectionClient,
secret: ReadonlyArray<string>
): void {
const knownKeys = introspectStateKeys(stateSchema);
const unknown = knownKeys === undefined ? [] : secret.filter((n) => !knownKeys.has(n));
Comment thread
jhoffner marked this conversation as resolved.
Resolved
if (unknown.length > 0) {
const valid = [...knownKeys!].sort().join(", ") || "(none)";
throw new Error(
`${definer} for "${ref}": secret names key(s) not on stateSchema: ` +
`${unknown.join(", ")}. Valid keys: ${valid}.`
);
}
const exposed = (client?.expose as ReadonlyArray<string> | undefined)?.filter((n) => secret.includes(n)) ?? [];
if (exposed.length > 0) {
throw new Error(
`${definer} for "${ref}": client.expose names secret key(s): ${exposed.join(", ")}. ` +
`A secret field never leaves the server; remove it from one list.`
);
}
}

/**
* `state` without the fields `config.secret` names — what every read that
* leaves the server starts from. Returns `state` itself when nothing is
* secret, so the common path allocates nothing.
*/
export function omitSecretFields<T extends JsonObject>(
Comment thread
jhoffner marked this conversation as resolved.
Resolved
config: { secret?: ReadonlyArray<string> },
state: T
): JsonObject {
Comment thread
jhoffner marked this conversation as resolved.
Resolved
const secret = config.secret;
if (secret === undefined || secret.length === 0) return state;
return omit(state, secret);
}

/**
* The client view of `state`: `config.client`'s projection run on the full
* state, then any `secret` field dropped from the result's top level. A
* `client.data` function sees the whole state (its parameter type says so) and
* can derive from a secret, as in `hasKey: s.value !== ""`; it cannot pass one
* through under its own name. Every engine read that sends a projection to a
* client goes through here.
*/
export function resolveClientData<TState extends JsonObject>(
config: {
client?: ResourceClientConfig<TState> | CollectionClientConfig<TState>;
secret?: ReadonlyArray<string>;
},
state: TState
): JsonValue | Promise<JsonValue> {
const projected = resolveClientProjection(config.client, state);
return projected instanceof Promise
? projected.then((value) => withoutSecrets(config, value))
: withoutSecrets(config, projected);
}

function withoutSecrets(config: { secret?: ReadonlyArray<string> }, value: JsonValue): JsonValue {
const isObject = typeof value === "object" && value !== null && !Array.isArray(value);
return isObject ? omitSecretFields(config, value as JsonObject) : value;
}

function pick<T extends JsonObject>(state: T, keys: ReadonlyArray<string>): JsonObject {
const out: JsonObject = {};
for (const key of keys) {
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/helpers/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ export {
getTransientKeys,
stripTransientKeys,
} from "./transient-slot";
export { resolveClientProjection, hasClientProjection, validateClientProjection } from "./client-projection";
export { resolveClientProjection, hasClientProjection, validateClientProjection, omitSecretFields, resolveClientData } from "./client-projection";
export {
introspectStateKeys,
getZodTypeName,
Expand Down
12 changes: 11 additions & 1 deletion packages/core/src/types/resource-collection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,15 @@ export type ResourceCollectionConfig<TState extends JsonObject = JsonObject> = {
* single-resource contract (the write tool gates on `llmWritable` alone).
*/
llmWritable?: boolean;
/**
* State fields that never leave the server. Every read the engine serves
* omits them: the client snapshot, the list and get-state routes, live
* deltas, and the debug routes, which otherwise return full state. A
* `client.data` function sees the full state, but a secret field in what it
* returns is dropped. Blocks on the server read them through the handle.
* A field named here cannot also be named in `client.expose`.
*/
secret?: ReadonlyArray<keyof TState & string>;
Comment thread
jhoffner marked this conversation as resolved.
Resolved
/**
* Allow blocks to mutate instance state (`patchState` / `setState` /
* `updateState` / `upsert` patch) and instance content (`writeContent`),
Expand Down Expand Up @@ -423,7 +432,8 @@ export function defineResourceCollection<
ref: config.pattern,
kind: "collection",
stateSchema: config.stateSchema,
client: config.client as Parameters<typeof validateClientProjection>[0]["client"]
client: config.client as Parameters<typeof validateClientProjection>[0]["client"],
secret: config.secret
});

validateReactTo("defineResourceCollection()", config.reactTo);
Expand Down
12 changes: 11 additions & 1 deletion packages/core/src/types/resource.ts
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,15 @@ export type ResourceConfig<TState extends JsonObject = JsonObject> = {
contentTemplateRef?: string;
llmReadable?: boolean;
llmWritable?: boolean;
/**
* State fields that never leave the server. Every read the engine serves
* omits them: the client snapshot, the list and get-state routes, live
* deltas, and the debug routes, which otherwise return full state. A
* `client.data` function sees the full state, but a secret field in what it
* returns is dropped. Blocks on the server read them through the handle.
* A field named here cannot also be named in `client.expose`.
*/
secret?: ReadonlyArray<keyof TState & string>;
Comment thread
jhoffner marked this conversation as resolved.
Resolved
/**
* Removed. `dynamic` never had an effect — collections are declared with
* `defineResourceCollection`. Typed `never` rather than deleted because
Expand Down Expand Up @@ -568,7 +577,8 @@ export function defineResource<
ref: config.ref ?? "(unnamed)",
kind: "single",
stateSchema: config.stateSchema,
client: config.client as Parameters<typeof validateClientProjection>[0]["client"]
client: config.client as Parameters<typeof validateClientProjection>[0]["client"],
secret: config.secret
});

// Single resources have no create/delete lifecycle — only state and content
Expand Down
13 changes: 13 additions & 0 deletions packages/core/test/client-projection.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import { describe, expect, it } from "vitest";
import { z } from "zod";
import {
omitSecretFields,
resolveClientProjection,
validateClientProjection,
} from "../src/helpers/client-projection";
Expand Down Expand Up @@ -60,6 +61,18 @@ describe("resolveClientProjection", () => {
});
});

describe("omitSecretFields", () => {
it("drops the fields a config names secret and keeps the rest", () => {
expect(omitSecretFields({ secret: ["value"] }, { value: "k", setAt: "t" })).toEqual({ setAt: "t" });
});

it("returns the same state object when nothing is secret", () => {
const state = { value: "k" };
expect(omitSecretFields({}, state)).toBe(state);
expect(omitSecretFields({ secret: [] }, state)).toBe(state);
});
});

describe("validateClientProjection", () => {
const stateSchema = z.object({ a: z.string(), b: z.number() });

Expand Down
Loading
Loading