Skip to content

feat(mcp): forward hooks_timeout, min_confidence and sort_by_length on laya_predict_batch - #766

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:pr94
Oct 1, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:pr94

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

laya.mcp.tools.laya_predict_batch and its server wrapper laya_predict_batch_tool now accept and forward three per-call controls that Router.predict_batch has always taken: hooks_timeout, min_confidence, and sort_by_length.

Why

Router.predict_batch grew these knobs (#577 for hooks, #583 for token budgets, #654 for sort-by-length) but the MCP layer only ever forwarded batch_size. A caller using the MCP server cannot:

  • apply the abstention gate to a batch the way they can to a single prediction,
  • override the hook execution deadline for a batched run,
  • tell the runtime to bucket-sort requests by token length so padding waste is minimised.

Every one of those is reachable from laya_predict (single) or Router.predict_batch (in-process); only the MCP batch tool swallows them. This is the same shape as the forwarding gaps already fixed for CLI (--min-confidence, --sort-by-length) and for the demo server.

How it was verified

uv run python3 tests/test_mcp.py
# 773 passed, 0 failed
uv run python3 tests/test_cli.py
# 135 passed, 0 failed

The BatchRouter fixture now records the kwargs it was called with. Four new assertions in test_batch_predict check that hooks_timeout=5.5, min_confidence=0.75, and sort_by_length=True are forwarded with those exact values, and that a plain laya_predict_batch(BATCH_REQUESTS, router=...) call leaves all three out of the kwargs entirely (so old strict stubs keep working).

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main
  • Tests pass locally
  • Docs or examples updated when the public API changed

Fixes #

…t_batch

Gaps found: MCP batch tool omitted 3 per-call controls that Router.predict_batch accepts.
- Added hooks_timeout (float), min_confidence (float), sort_by_length (bool) as optional parameters
- Forward all three through to router.predict_batch() in tools.py
- Updated server wrapper to accept and pass them through
- Test suite validates forwarding with BatchRouter tracking kwargs
- Mutation-check: gate ensures these knobs reach Router.predict_batch
@NandhaKishorM
NandhaKishorM merged commit 7b106d7 into NandhaKishorM:main Oct 1, 2026
24 of 25 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged for 0.3.23.

Verified on the integrated branch, not on the PR in isolation: every suite in the ci.yml test lane plus the pytest lane green, compileall clean, and a decision diff against main on the real english and multilingual checkpoints (72 answers over choice, score and noul, short and long states, budgeted and predict_long) showing zero decision flips and a max probability delta of 0.000000.

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants