Repository navigation
fix(cli): judge the --sort-by-length note per checkpoint group, not per file (#1111) - #1114
Conversation
…er file (NandhaKishorM#1111) The note's gate read the file: it printed only when --batch-size could not clear 1 < chunk < len(lines). The reorder itself switches per checkpoint and question group inside Router.predict_batch, on each group's own state count (agent.py:1519, identically onnx_agent.py:761), so a file that auto-routes to several checkpoints reordered nothing while the note stayed silent -- the silent same-speed result the note exists to prevent (NandhaKishorM#1111). Router.predict_batch now takes an optional keyword-only diagnostics dict and, when sort_by_length is set, reports under "sort_by_length" how many question groups reached an agent, how many could reorder, and the smallest / largest group size -- counted where the groups are actually formed, so an agent whose predict_batch predates the knob (NandhaKishorM#294) reports as unsorted, not as an error. RemoteRouter.predict_batch accepts and ignores the same argument, keeping the signatures identical (test_hooks_api). run_batch asks for the report only from routers whose predict_batch names the parameter, notes after the run from what actually reordered -- keeping the pre-run wording, sized by the group, when one group saw the whole file -- and leaves routers without the parameter (stubs, **kwargs forwarders) exactly as they were: same call arguments, same pre-run note. The grouping semantics in the agents are untouched. The handwritten signature list on the hooks API quick-reference page gains the parameter too. Red (this tree's tests against 6880462 sources): tests/test_cli.py 190 passed, 2 failed -- the two no-op-silence assertions; tests/test_router.py died with TypeError: unexpected keyword argument 'diagnostics'; tests/test_hooks_api.py 547 passed, 1 failed. Green: 192 / 810 / 548 / 78 passed, 0 failed (plus confidence 214, lang-guess 17, blank-lang 24, evals-api 50), ruff and compileall clean.
|
Holding this out of 0.4.3, not closing. One repository gate catches it, and the fix is small but it is a design decision that should be yours rather than mine. Measured on the integration branch with this merged: That pin exists for this exact case. Its docstring: "A control added to The awkward part, and the reason I am not fixing it myself: Refusing rather than forwarding is clearly right, for what it is worth: The feature itself is good and the reasoning behind it is right. Judging the note per checkpoint and question group is where the reorder actually happens, and |
…ling and dropping it #1114 adds `diagnostics` to `Router.predict_batch` so the CLI can say which checkpoint groups actually reordered (#1111). The HTTP batch endpoint pins its control list against that signature, and a control has to be placed on one side of the line before the suite goes green: forwarded, lifted onto each item, or refused. It is refused, and it is not a `BODY_REFUSALS` member. Those five run inside the server process. This one is an out-parameter: `predict_batch` writes the report into a dict the caller already holds a reference to, and an HTTP caller holds none, so a body that sent `diagnostics` would be answered 200 while the report it asked for was filled in and discarded with the request. `BODY_REFUSALS` is also pinned against `Router.predict`, which has no `diagnostics`, so adding it there would red the single-endpoint pin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Merged, with one follow-up commit from me that the PR needed and did not have. The bug reproduces exactly as #1111 describes. Six lines, The old pre-run note checked What I had to add. I noted that on the previous round and it is still the reason this could not merge as submitted, so I wrote the serve side myself ( The reason it is a separate tuple rather than a The rest of the PR is right as written. 87/87 suites pass on the merged tree, and the 94-row real-weights decision diff against 0.4.3 is identical. |
What
--sort-by-length's no-op note is now judged where the reorder actually happens: per checkpoint/question group, not per file.Router.predict_batchtakes an optional keyword-onlydiagnosticsdict (laya/router.py:1602) and, whensort_by_lengthis set, reports under"sort_by_length"how many question groups reached an agent, how many of those could reorder, and the smallest / largest group size — counted atlaya/router.py:1799-1813, where the groups are actually formed.run_batch(laya/cli.py) asks for the report only from routers whosepredict_batchnames the parameter (_takes_predict_diagnostics,laya/cli.py:310— exact-name check, so a**kwargsforwarder cannot silently swallow the dict) and prints the note after the run from what actually reordered (_note_sort_report,laya/cli.py:328): the pre-run wording kept, sized by the group, when one group saw the whole file; a per-group accounting (laya/cli.py:345) when several groups could not.RemoteRouter.predict_batchaccepts and ignores the same argument in the same position (laya/mcp/remote.py:206), keeping the two signatures identical. The hooks API quick-reference signature list gains the parameter (docs/hooks/api.md:303).Untouched by design:
laya/cli.py:371-373still forwardssort_by_lengthonly when the user asked for it (the stub/old-agent compatibility contract), and the grouping window semantics in the agents (laya/agent.py:1519,laya/onnx_agent.py:761) are not modified. An agent whosepredict_batchpredates the knob (#294) reports as unsorted, not as an error.Why
Fixes #1111. The note's gate read the file:
chunk = args.batch_size ... else len(lines)andif not 1 < chunk < len(lines)decided whether the warning printed at all. But the reorder itself switches per checkpoint and question group insideRouter.predict_batch— each group calls its agent with the group's own state count against1 < chunk < len(states)(laya/agent.py:1519, identicallylaya/onnx_agent.py:761). When auto-routing splits a file across several checkpoints, the two numbers diverge: nothing reorders anywhere while the file-level1 < chunk < len(lines)holds, so the note stays silent — the exact silent same-speed result the note exists to prevent. The CLI cannot know the post-routing group sizes before the call, so the only honest place to judge is after the run, from the router itself.How it was verified
Plain-script suites as the repo's CONTRIBUTING/AGENTS.md prescribes (these are not pytest-collectable); every command under
timeout 300.Full suites, final tree (re-run at submission time, all exit 0):
Red → green (new tests only, sources stashed back to
6880462):tests/test_cli.py: 189 passed, 2 failed — exactly the two new no-op-silence assertionstests/test_router.py:TypeError: unexpected keyword argument 'diagnostics'at the new call — crash = redtests/test_hooks_api.py: 547 passed, 1 failed — the signature-contract test (diagnosticsparameter missing)Repo gates:
Honest declarations:
timeout 300 pytest -k "sort_by_length or sort"was not run as written: nopytestin this environment (PEP 668 externally-managed interpreter; declined--break-system-packages), and per the repo's convention these suites are plain scripts — pytest collects no cases from them. The documentedpython3 tests/test_xxx.pycommands above are what was run instead.tests/test_mcp.py= SKIP (mcp extra not installed; the repo's own established skip, exit 0).tests/test_mcp_remote.pynot run (needs pytest + mcp extra); instead theRemoteRouter.predict_batch↔Router.predict_batchsignatures were compared parameter-by-parameter with a direct script, confirmingdiagnosticsis accepted and left untouched (the 548-greentest_hooks_apicovers the same contract).Router.predict_batch's grouping contract with a repo-style stub (/tmp/repro1111.py), and an end-to-end script with a realRouter+ stubbedload(/tmp/e2e1111.py) verifies the full chain prints the per-group note: ALL OK.Disclosure: this change was prepared by an AI-assisted contributor (Claude Code), consistent with the claim comment on #1111.
Checklist
mainFixes #1111