Code map · Desktop scenarios · Agent/Worker scenarios
Choose a user promise, then the layer that can observe its failure. Keep tests beside that owner and preserve independent assertions. A focused filesystem conflict test can document a product guarantee more precisely than another broad happy-path UI journey.
| Layer | Appropriate assertion | Evidence limit |
|---|---|---|
| Pure state / policy | Intent, thresholds, branch isolation and request construction | No service dispatch or production wiring |
| Component with real filesystem/database/controlled service | Ownership, rollback, authorization, cancellation and recovery | Not every production entry or OS |
| Desktop React + mock IPC | Controls, edits, pending/error feedback and emitted commands | No Tauri effects, native clipboard or actual daemon mutation |
| Desktop command authority / native compilation | ACL check, configured Mac desktop-pilot --all-targets consumer compilation |
An allowed/compiled command is not a WebView invocation or native scenario |
| Real CLI/daemon or Gateway listener | Published plans, actual upstream attempts and durable effects | Controlled upstreams do not establish live-model answer quality |
| Native Desktop / installed Agent / live provider | The selected real integration path and independently observed outcome | Bound to its exact environment, revision and executed scenarios |
| Frozen compatibility / expected-red / oracle fixture | A historical reader, intended rejection or test oracle | Not current product completion |
Start with the representative cases in the capability map. A file
named product, a golden containing green, or a zero process exit does not prove
execution. In tools/product-e2e, transaction recovery
contains expected-red composition, adapter contracts
are component evidence, and oracle tests
validate the oracle. The real installed management loop is in
publication_process.rs.
| Need | Reuse / preserve |
|---|---|
| Prepare a daemon's current sources, grants and plans | Publication Product: its Product.cli is internal Local Control notation, not the standalone CLI; protected input belongs to that fixture's daemon |
| Configure a real native Agent | Agent product support: shared Preview → Apply → status and bounded process capture; native argv/ownership/result interpretation remain in the leaf |
| Prove saved model configuration | Additional-model journey: ordinary native startup reads saved settings, without injected provider/endpoint/credential overrides |
| Observe native routing and rejected attempts | NativeContextUpstream: count every model POST, including rejected endpoints/models/credentials; keep the ledger for positive route/history assertions |
| Prove exact Continue, concurrent routes and cancellation | Native context, boundary journey, boundary oracle guide |
| Prove restoration of old durable state | Frozen producer fixtures, crash-boundary tests: old bytes through the current consumer, not a new serializer pretending to be history |
| Preserve native probe failure classification | Pi probe tests, immutable launcher: mutable test bodies are interpreter inputs; executable availability and version/output failures have separate assertions |
| Exercise actual Desktop controls | Browser scenario map and native acceptance: select stable scenario IDs and require every declared outcome |
The publication fixture leases its listener address across daemon restarts; do
not substitute a released bind(0) probe. Its protected-input channel cannot be
attached to an existing Desktop. Prepare that instance first, then use real UI
mutations. Main-Agent delegation must separately invoke the actual public CLI.
Keep borrowed user resources, writable native acceptance configuration, fresh product storage and per-run materials separate. Each product instance needs a private runtime/storage namespace. CLI submissions have no local receipt directory; recovery uses the original key and request, with title/detail lookup only as an aid. CLI submission regressions cover denied local writes and uncertain responses; durable admission and native Main journeys remain separate. Expose the selected CLI and Pi's selected Node consistently to main-Agent checks and Worker launch. Never repair permissions on pre-existing user directories to satisfy a fixture; cleanup verifies the owned object's identity and preserves foreign files.
For a native run, read the ecosystem's acceptance entry for explicit inputs, writable roots, required outcomes and cleanup. Report a preparation/cleanup failure separately while retaining the first product failure. Process exit, successful setup or the model's success text cannot replace a Gateway receipt and an independent tool artifact.
Build legal current state through current producers; compute expected outcomes independently. A historical journal fixture must retain its original producer bytes, including relevant feature combinations such as JSON ordering. Register recovery readers and removal conditions in the compatibility registry.
Require the capability under test rather than a fixed total number of Agents. Keep inventory completeness at the registry boundary: registry, bundle source and bundle producer must agree. Changed deterministic generators run twice; the second run must leave zero diff.
Use shared enable/adjust/disable and blocked-action recovery scenarios from the Agent feature guide. Native protocol differences alone do not justify an extra verification control. Browser fixtures, including standalone editors, select stable IDs rather than positional slices or translated labels; require nonempty selection and all expected outcomes.
Before consolidating tests, list their unique failure conditions. Preserve crash, stale-preview, cancellation, foreign-file, authorization, timeout and compatibility cases even when happy paths overlap. For a broken gate, demonstrate that a realistic bad case fails. For managed native overrides, retain a positive control showing that the conflicting native setting would otherwise be consumed.
Protocol changes reuse saved-route and native-context journeys across Responses and Messages. Assert actual request paths, trusted receipt identity, rotation and independent restore. A removed-route witness uses its previously working protocol and proves a typed authorization rejection plus zero upstream requests; an unknown HTTP path or old restore fixture does not prove route revocation or native execution.
- Run
python3 scripts/test-plan.py --base <exact-base>; use--integrationfor branch integration. Review affected consumers and explain semantic adjustments. - Follow the selected commands and CI definitions.
Use
scripts/validation.pyfor configured hosts, exact committed Rust candidates, feature plan, phase and related runs. Compile affected native consumers early; featureless Desktop checks and zero-selected tests do not satisfy that gate. - Diagnose focused failures, freeze after affected checks pass, then complete the selected final checks once. Missing required cases remain unexecuted, not green.
- Record revision, command, platform, cases, result and evidence location. Preserve reused evidence at its original SHA; assess the three slowest measured modules once without automatically refactoring the frozen candidate.
Final backend DAG admission and preparation are owned by validation-schedule.py, with regressions. CPA bridge process fixtures run in an isolated lane before other listeners; Gateway runtime consumers wait for prepared product roles.
Code-map and local ownership-guide edits require link/owner review and the selected contract checks, not new model calls. Runtime-embedded Markdown and shipped Skills can change behavior and need their actual consumers. Instructions and compilation remain distinct from executed business evidence.