Repository navigation
Preserve capability intent when checking and saving a model connection - #433
nillikechatchat wants to merge 4 commits into
Conversation
… save Reconstruct saved local capability facts from stored values and bases so a label-only edit keeps unknown optional capabilities, explicit values survive restart, and an exact same-ID catalog candidate no longer replaces the saved local row. Separate editor suggestions from the facts serialized by Check and Save in the Desktop model-connection flow. Suggestions stay in editor context, explicit field edits and Apply suggestions declare UserDeclared values, clearing yields unknown, and selection rows report catalog membership separately from this check's directory observation. Extend the production model-connection product journey with a controlled directory covering an observed-only eligible ID, label-only and explicit field edits including false, explicit adoption, partial directory and a per-model tool test. Signed-off-by: nillikechatchat <yuanhenglizhen2050@163.com> Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
Replace the closure that used an inferred generic placeholder with a generic function so one helper checks bool, u64 and native reasoning facts without a type mismatch in the daemon test build. Signed-off-by: nillikechatchat <yuanhenglizhen2050@163.com> Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
Wrap the multi-ID test loop and the membership comparison so `cargo fmt --check` passes on the daemon crate. Signed-off-by: nillikechatchat <yuanhenglizhen2050@163.com> Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
The capability-intent product scenario asserted that an inference-only Check reports an available directory and observed model IDs. That is not what the product does: an inference-only Check (inference_model_id set) short-circuits before the directory probe, records a missing-credential observation with no issues, and therefore reports directory `not_run` with no observed IDs. The assertion encoded an incorrect expectation rather than a product defect. Assert the actual contract instead: directory stays `not_run`, no model ID is observed, no directory GET is issued, and the single POST probe carries the selected model plus tools. This also proves the directory result is independent of the inference evidence, which is the property the scenario is meant to demonstrate. Signed-off-by: nillikechatchat <yuanhenglizhen2050@163.com> Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
|
|
||
| /** Suggestions stay in the editor; the wire model carries only the draft's declared facts. */ | ||
| export function toWireModel(model: ModelDeclaration): Omit<ModelDeclaration, 'client_id' | 'suggested_capabilities'> { | ||
| const { client_id: _clientId, suggested_capabilities: _suggestions, ...wire } = model; |
There was a problem hiding this comment.
[P1] Keep editor-only state out of Check requests.
toWireModel forwards cleared_capabilities, including [] after a regular field edit. Both Check paths use this serializer, and the shared NativeUserModelDeclarationV1 rejects unknown fields. I captured requests from the production form after editing a field, clearing one, and applying suggestions. The v0.6.0 decoder returned INVALID_ARGUMENTS for all three. Removing only that field let the requests reach transport.
Serialize the wire fields explicitly and exercise these three actions through the strict decoder for both registered and custom Check requests.
| suggestions: ModelCapabilities | undefined, | ||
| ): ModelDeclaration { | ||
| if (!suggestions) return model; | ||
| return model.suggested_capabilities ? model : { ...model, suggested_capabilities: suggestions }; |
There was a problem hiding this comment.
[P2] Preserve the selected template's suggestions.
blankModel already sets generic suggestions, so withSuggestions discards the supplied capability_prefill. With the bundled Bailian Token Plan data, the qwen3.8-flash template offers 1,000,000 context tokens and 131,072 output tokens, while the editor displays 200,000 and 32,768. Applying those values would record the generic defaults as user declarations.
Use the applicable template suggestions when constructing the draft, and verify what the editor displays and adopts with a bundled template.
| const canSelect = modelCanBeSelected(result, model, true); | ||
| return <div className="model-result-row" key={model.model_ref}><label className="check-row"><input type="checkbox" data-autofocus={index === 0 || undefined} disabled={!canSelect} checked={canSelect && selected.has(model.model_ref)} onChange={event => { const next = new Set(selected); if (event.target.checked) next.add(model.model_ref); else next.delete(model.model_ref); selectionRef.current = { result, refs: next }; setSelected(next); setError(''); }} /><div><strong>{model.display_name}</strong><span>{draft.display_name}{model.fact_basis === 'runtime_fallback' ? ` · ${zh ? '运行时保守兜底' : 'Conservative runtime fallback'}` : ''}{!canSelect ? ` · ${modelAvailabilityMessage(model.reason, zh)}` : ''}</span></div></label>{custom && !canSelect && <button className="btn btn-quiet" type="button" onClick={() => { | ||
| const observation = modelObservationMessage(result, model, zh); | ||
| return <div className="model-result-row" key={model.model_ref}><label className="check-row"><input type="checkbox" data-autofocus={index === 0 || undefined} disabled={!canSelect} checked={canSelect && selected.has(model.model_ref)} onChange={event => { const next = new Set(selected); if (event.target.checked) next.add(model.model_ref); else next.delete(model.model_ref); selectionRef.current = { result, refs: next }; setSelected(next); setError(''); }} /><div><strong>{model.display_name}</strong><span>{observation ? `${observation} · ` : ''}{draft.display_name}{model.fact_basis === 'runtime_fallback' ? ` · ${zh ? '运行时保守兜底' : 'Conservative runtime fallback'}` : ''}{!canSelect ? ` · ${modelAvailabilityMessage(model.reason, zh)}` : ''}</span></div></label>{custom && !canSelect && <button className="btn btn-quiet" type="button" onClick={() => { |
There was a problem hiding this comment.
[P2] Show official catalog match alongside directory observation.
This row renders the current directory observation without using membership. In a partial-directory browser fixture, the catalog-matched row and the observed-only row both showed “Returned by this check” with no indication of their different origins. A row with a local display name also hid its exact upstream ID.
Render the catalog match from membership independently of the observation label, and show the upstream ID when the label differs. This is part of the Phase 1 selection contract in #411.
| ] as const).map(([key, label]) => <label className="field" key={key}><span className="field-label">{label}</span><input className="input" type="number" min="1" value={reasoning[key]} onChange={event => updateCapability('native_reasoning', { value: { ...reasoning, [key]: Number(event.target.value) }, basis: 'user_declared' })} /></label>)} | ||
| </div>} | ||
| </Disclosure> | ||
| {hasSuggestions && <div className="manual-capability-suggestions"><button className="btn btn-quiet" type="button" onClick={() => onChange(applySuggestedCapabilities(model))}>{zh ? '采纳建议值' : 'Apply suggested values'}</button><span className="field-help">{zh ? '建议值仅填充仍为“未知”的能力,不会覆盖你的填写。' : 'Suggestions fill only values still marked Unknown and never overwrite your edits.'}</span></div>} |
There was a problem hiding this comment.
[P2] Show the suggestions before they can be adopted.
I could click “Apply suggested values” with “Capabilities and limits” collapsed, adopting values I had not seen. Phase 1 calls for explicit adoption of the displayed suggestions. Put the action alongside the visible values and indicate which fields still contain pending suggestions. The regression should also confirm that adoption preserves fields the user edited or cleared.
| model: Pick<CandidateModelView, 'upstream_model_id'>, | ||
| zh: boolean, | ||
| ): string | null { | ||
| const observed = new Set(result.observed_model_ids ?? []); |
There was a problem hiding this comment.
[P3] Build the observed ID set once per Check result.
The list calls this helper for each candidate on each render, including when the user toggles a selection. At 1,000 returned IDs and 1,000 candidate rows, that is about one million set insertions per render. In a local measurement, label computation alone took about 19 ms per pass at that size. Reuse the set across rows and selection changes.
| local_model('field-edit-model', 'Field Edit Model', edited), | ||
| local_model('adopted-model', 'Adopted Model', adopted_caps), | ||
| ] | ||
| draft = native_check_draft( |
There was a problem hiding this comment.
[P2] Exercise the same observed-only ID through registered Save and recheck.
The directory step proves that gpt-5.4-mini is selectable. The save step then uses three different, manually declared IDs on a custom connection. TASK-411004 calls for the registered production path. Use one eligible observed-only ID throughout Check, label edit, Save, restart, and saved-source recheck. Verify the stored and reconstructed values and bases, including unknown fields and an explicit false.
Problem
A newly observed upstream model ID can be selected, but editing and rechecking it replaced unknown optional capabilities with generic values labeled as user declarations. A same-ID catalog candidate could also replace a saved local row, and selection rows did not distinguish official catalog membership from a return by the current directory check. The result was a saved model that claimed facts the user never declared.
Result
Phase 1 of Design #411 keeps capability intent consistent across Check, Save, restart and saved-source recheck.
falsestays known, and an exact same-ID catalog candidate no longer replaces the saved local row. Qualification and denial still apply by ID during candidate generation.UserDeclared; clearing a field yields unknown and is remembered so a later adoption does not refill it.upstream_model_idinobserved_model_idsmarks a return from this check, a completed reachable directory distinguishes Not returned by this check, and partial, unavailable, invalid or omitted directories use Not confirmed by this check.false, explicit adoption, partial directory and a per-model tool test.Phase 1 keeps the released
ComputeCandidateViewV2and existing Check request/response shapes. Itsdeny_unknown_fieldscontract is preserved, so field-level projection, price and free qualification stay in TASK-411001.Compatibility
fact_basissummary; the public view projects one basis per model, so mixed local facts surface asunknownthere while durable recheck compares decoded per-field values and bases.Validation
Executed in this checkout:
On
a860a286:scripts/test-contract-convergence.py: 23 tests passedscripts/check-contract-convergence.py: green, 42 subjects and 37 legacy entriespython3 -m py_compileand import of the extended product scriptcd apps/desktop && node --experimental-strip-types --test tests/*.test.mjs: 465 passed, 0 failedcd apps/desktop && ./node_modules/.bin/tsc --noEmit: passedOn
c87c03a7and94c657c8:cargo check -p hiroute-daemon --lib --tests(0 errors) andcargo fmt --all --checkpass.94c657c8:backend-shard (integration-daemon)and(integration-product), plus quality, gates, integration-foundation, integration-smoke, integration-gateway, frontend, DCO, scope, selection and check all pass.Known gaps:
backend-shard (unit)fails oncontrol::agent_removal::tests::shared_mutation_excludes_teardown_and_teardown_refuses_both_apply_entries(crates/daemon/src/control/agent_removal.rs:96,DaemonUnavailablevsInvalidArguments). This is a pre-existing failure on canonicalmainatbf99f0f6with the same assertion and values, in a file absent from this branch and outside the changed paths. A maintainer rerun ofbackend-shard (unit)is requested to confirm the flake.real_process_native_model_save_publishes_and_reaches_gatewaywas not executed locally because buildingtarget/debug/hiroutedexhausted the available disk; the integration shards cover this path in CI.Docs
issue-spec/specs/model-connections/spec.md: added the suggestion/declaration requirement and extended the saved-source recheck requirement.apps/desktop/src/features/model-connections/README.md: updated for the suggestion and observation semantics.Closes #432
Refs #411
Refs #403