Repository navigation
feat(mcp): expose core's min_confidence abstention control on the decision tools - #657
Conversation
…ision tools min_confidence (NandhaKishorM#361) is a validated keyword of Agent.predict, Router.predict/ predict_batch and laya.decide on main, but the MCP layer never forwarded it: a client could pin a checkpoint and size its token budget over the wire yet could not ask Laya to abstain on an unsure answer. laya_predict, laya_shortlist, laya_preset and laya_decide now take it and forward it to core, validated in the tool layer via laya.confidence .check_min_confidence so a bad value is an invalid_min_confidence ToolError rather than a forward pass that never runs. Only set values are forwarded, so a no-abstention call stays byte-identical to one before the keyword existed. On laya_decide a field below the threshold comes back null in values while its confidence survives -- parity with laya.decide(min_confidence=...).
|
The only failing required check here is
This PR changes no dependency file, and all three packages arrive transitively through the crewai / langgraph / llama-index-core extras, so there's no pin here to bump. Even if there were, 6 of the 7 rows have no fix to move to. Since Your call on the fix, since I'd rather not loosen a security gate from a contributor PR: either add ignore entries for the unpatched IDs and bump |
`env_device()` is what `laya/mcp/server.py` hands to `Router(device=...)`, and it returned
the environment value with only surrounding whitespace stripped. `resolve_device()`
lower-cased the same value, and that is what `laya_status` reports.
torch's device parser is case-sensitive. Measured with torch 2.13.0+cpu:
```
LAYA_DEVICE env_device reported torch.device(env_device) torch.device(reported)
'CPU' CPU cpu REJECTED ok
'CUDA' CUDA cuda REJECTED ok
'CUDA:0' CUDA:0 cuda:0 REJECTED ok
'cpu' cpu cpu ok ok
'cuda:0' cuda:0 cuda:0 ok ok
```
So for one environment, `laya_status` reported a device as configured and working while the
Router was being built with a value torch refuses. The failure surfaced as
`ToolError("internal_error", "router construction failed: ...")`, which classifies a
misconfigured server environment as a server fault rather than a bad setting.
`env_device()` now normalises the device type to lower case, and `resolve_device()` returns
that same string instead of lower-casing a second time, so the value handed to torch and the
value reported are one string by construction. Splitting on the first colon keeps the index
intact -- `CUDA:0 -> cuda:0` -- because torch wants the type lower-cased and does not care
about the index.
Unset and whitespace-only still mean "not asked for" and still let `resolve_device()`
auto-detect; that difference is the point of the two functions and is unchanged. Blank
input, padding, and an explicit `resolve_device(force=...)` keep their behaviour.
The existing test asserted the old contract by name (`device/env_raw_for_torch`, with a
comment saying the value goes "verbatim to torch"). That contract is what rejected
`CUDA`, so the assertion is updated to the new one and extended to cover the case that
motivated it, rather than left passing on a value that happens to be lower-case already.
`README.md:929` documented the old wording ("the value is passed straight to torch") and now
states the normalisation.
```
python tests/test_mcp.py 683 passed, 0 failed (was 679)
python -m compileall -q laya tests
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
git diff --check
```
Not run here, and not in this diff: `tests/test_serve.py` exceeded a 400 s timeout in this
environment, and `tests/test_mcp_local_e2e.py` needs a real checkpoint from the Hub
(`ModuleNotFoundError` for `laya` with no install metadata). Both are pre-existing
environment limits rather than results, so I am flagging them instead of implying they passed.
`laya/mcp/device.py` is touched by no open PR. Four open PRs edit `tests/test_mcp.py`
(#657, #640, #636, #445) and eighteen edit `README.md`; I read the patches and none adds or
removes the lines changed here -- #640 only has the device import as surrounding context,
#387 matches `LAYA_DEVICE` without changing that row, and #53 adds an unrelated mention.
Found during a cross-repo contract review of how a configuration value is normalised across
the entry points that read it; the defect and the fix are this repository's own.
|
Passed checks and merged. Thank you @aashish254! It ships in 0.3.22. |
What
min_confidence-- core's opt-in abstention gate from #361 -- is now a keyword on the MCP single-call decision tools (laya_predict,laya_shortlist,laya_preset,laya_decide). It is validated in the tool layer and forwarded to the answering forward pass; onlaya_decidea field whose answer falls below the threshold comes backnullinvalueswhile its raw confidence survives.Why
min_confidenceis already a validated keyword ofAgent.predict,Router.predict/Router.predict_batchandlaya.decideonmain-- the 0.3.21 tagline itself ships "opt-in abstention ... on every surface". Butgrep -c min_confidence laya/mcp/is0: the MCP layer never forwarded it. #567 carried every other per-call control (task/lang/max_len/head_max_len) across to these tools, so a client over MCP could pin a checkpoint and size its token budget, yet could not ask Laya to abstain on an unsure answer -- the documented capability had no reachable lever on this surface.Concrete use case: a caller that pipes
laya_decide'svaluesstraight into an action (route a ticket, set a priority) needs "not confident" to be distinguishable from a confident pick, without parsing theconfidencemap itself. Over MCP that was impossible before this change.How it was verified
Before/after measured on
main(9d95567):laya.mcp.tools.laya_predictparams[..., 'head_max_len', 'router', 'agent'](nomin_confidence) -> now[..., 'head_max_len', 'min_confidence', ...], while corelaya.decide/Router.predict/Router.predict_batchalready carry it -- so this closes a surface gap, not a core behavior change.test_min_confidence_control()drives it at the seam: a recordingControlRouterproves the kwarg is forwarded only when set (a no-abstention call reaches core exactly as before),laya_decide(min_confidence=0.9)keeps the 0.94-confidence field and nulls the 0.8/0.89 ones withconfidenceintact and byte-parity tolaya.decide(..., min_confidence=0.9), and bad values (-0.1/1.1/True/"0.5"/nan/inf) becomeinvalid_min_confidencebefore any forward pass runs. Three faithful mutants (drop the predict forward, drop the decide forward, neuter the validator) were each killed by a named assertion.Checklist
mainOverlaps note: this touches
laya/mcp/tools.py,laya/mcp/server.pyandtests/test_mcp.py-- the same files my open #640 (batch-request token budget) and #636 (shortlistkdefault) touch, but in different functions (laya_predict/laya_shortlist/laya_preset/laya_decidevs the*_batchandksurfaces), so they merge independently. No open PR claims MCPmin_confidence; #644 is thelaya-tsport of #361, a different surface.Fixes #