Skip to content

fix(integrations): a delegation choice naming no agent is refused, not indexed - #1416

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:integrations-choice-index
Oct 29, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:integrations-choice-index

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

LayaCrewRouter.route, LayaAgnoRouter.route and LayaSingleSelector._select parse the remote
choice string (agent_N / choice_N) into an index and use it directly on the caller's list. They
now check that index against the candidates before indexing, and raise the same
Malformed decision response RuntimeError the package already raises for a choice of the wrong
type.

  • laya/integrations/crewai.py:412
  • laya/integrations/agno.py:594
  • laya/integrations/llamaindex.py:402

Why

chosen_key is arbitrary JSON from a Laya server (or any local runner's predict return) and
agents/choices is the caller's list, so the two have no reason to agree. Nothing connected them:

$ python /tmp/u6_repro.py <base worktree>        # 2 agents, choice="agent_9"
crewai agent_9         -> IndexError: list index out of range
crewai agent_-1        -> NO ERROR, index=-1
llamaindex choice_9    -> IndexError: list index out of range
llamaindex choice_-1   -> NO ERROR, index=-1

Two failure modes from one unvalidated parse. Past the end, agents[chosen_idx]
(crewai.py:441, agno.py:625, llamaindex.py:422) raises a bare IndexError out of the path whose
job is to fail closed -- and it is a leak, not a check: the line above it already refuses a choice
that is not a string by name. Below zero, Python's negative indexing wraps, so agent_-1 silently
delegates to the last agent and the decision carries agent_index=-1, which is wrong for the
caller too. tools.py:478/:493 already handles the same shape in the sibling adapter -- it checks
membership in the candidate lookup and answers with Unknown or missing tool decision ... not in candidate tools -- and LayaQueryRouter.route in llamaindex.py:703 treats a name outside its
engines as no match. These three routers were the only ones that indexed instead.

The criteria dict is built from exactly the list being indexed (_format_agent_criteria,
_format_choices_criteria both key agent_{i}/choice_{i} over enumerate(agents)), so "the parsed
index is outside the list" is precisely "the decision named a candidate that does not exist".

After:

$ python /tmp/u6_repro.py <this worktree>
crewai agent_9         -> RuntimeError: Malformed decision response from Laya server at http://x: decision 'agent_9' names agent 9, which is not in the 2-agent crew
crewai agent_-1        -> RuntimeError: Malformed decision response from Laya server at http://x: decision 'agent_-1' names agent -1, which is not in the 2-agent crew
llamaindex choice_9    -> RuntimeError: Malformed decision response from Laya server at http://x: decision 'choice_9' names choice 9, which is not among the 2 choices
llamaindex choice_-1   -> RuntimeError: Malformed decision response from Laya server at http://x: decision 'choice_-1' names choice -1, which is not among the 2 choices

The highest real index still routes (a range check, not a rejection of agent_3), which is asserted
as a control in each suite.

How it was verified

Each adapter's suite grows by the same 13 checks -- 4 bad indices x (raises / names the choice / names
the count / is a RuntimeError) plus the last-index control. Totals were read out of the suite's own
PASS/FAIL lists with runpy, because the file prints its PASS: summary before these sections:

python -B -c "$(run the suite with runpy, print len(PASS), len(FAIL))"
# base   : test_crewai PASS=238 FAIL=0 | test_agno PASS=229 FAIL=0 | test_llamaindex PASS=256 FAIL=0
# this   : test_crewai PASS=251 FAIL=0 | test_agno PASS=242 FAIL=0 | test_llamaindex PASS=269 FAIL=0

The guard itself is what those checks test -- with the three if not 0 <= ... blocks removed (base
laya/integrations/*.py dropped back in), every suite exits non-zero and names the leak, so no check
is a no-op:

  - choice-index/agent_4/route is RuntimeError IndexError
  - choice-index/agent_-1/route raises no error raised (routed to index -1)
  - choice-index/choice_9/select is RuntimeError IndexError

The files were then restored byte-identically (cmp against the saved copies) and the suites re-run
from disk.

ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120   # All checks passed!
python -m compileall -q laya/ tests/                                  # rc=0
python laya-dotnet/tools/dump_routing_golden.py --force --out /tmp/g   # no file differs from the
                                                                        # committed goldens
python laya-java/scripts/gen_fixtures.py --check                       # lang_detect.json /
                                                                        # python_json.json stale,
                                                                        # identical to base
python laya-ts/scripts/option_stability_fixtures.py --check            # rc=0
git merge-tree --write-tree fe9a6c35 dd3e15b7                          # clean

No port fixture or golden moves: the generators that could pin this (laya-dotnet/tools/*_cases.py,
laya-rust/tools/, laya-java/scripts/gen_fixtures.py, laya-go/scripts/gen_go_fixtures.py,
laya-ts/scripts/option_stability_fixtures.py) contain no crewai/agno/llamaindex decision case,
and Malformed decision response appears in no port fixture -- the same sweep that let #1335 and
#1403 through.

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main
  • Tests pass locally
  • Docs or examples updated when the public API changed -- no public API changed: the routers'
    signatures, return types and existing error family are untouched, and docs/crewai.md, docs/agno.md
    and docs/llamaindex.md do not enumerate the malformed-decision refusals this joins, so there is no
    list to add it to

Fixes #

…, not indexed

`chosen_key` is remote JSON and `agents`/`choices` is the caller's list, but `agent_N` was parsed
straight into `agents[N]`. `agent_9` for a two-agent crew raised a bare IndexError out of the
routing decision, and `agent_-1` wrapped to the LAST agent -- a route no caller asked for and
nothing in the decision names. `tools.py` already answers an unknown decision by name instead of
raising; the three routers now check the index against the candidates they built `criteria` from.
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged in 0.4.18. The negative index is the reason this is more than an error-message improvement, and it deserves stating plainly: chosen_key comes from parsed remote JSON, and agent_-1 silently selected the LAST agent. A wrong-but-plausible delegation is worse than a crash, because nothing downstream can tell it happened.

agents[chosen_idx] raising a bare IndexError out of a routing decision was the other half, and 0 <= chosen_idx < len(agents) covers both with one check.

Your justification is the right one: criteria is built from exactly these agents, so an index it does not name is a candidate that does not exist. That makes this a malformed decision rather than an out-of-range lookup, which is why routing it through _malformed alongside the other unknown-decision refusals is correct and not just convenient.

Fixing all three integrations in one change is right for the same reason as the BOM work: agno, crewai and llamaindex share this shape, and leaving one of them wrapping to the last agent would be harder to find than a uniform refusal. The messages differing by domain word (team, crew, choices) is a nice touch.

For the record, since the automated boundary scan flagged it and someone may re-run that scan later: the _LLAMA_CHOICE_INDEX constant in tests/test_llamaindex.py matches a pattern I grep for on every merge. It is the pre-existing llamaindex integration's own naming and has nothing to do with the thing that pattern looks for.

Verified: 706 tests across 93 suites.

@NandhaKishorM
NandhaKishorM merged commit be69cf4 into NandhaKishorM:main Oct 29, 2026
63 checks passed
NandhaKishorM added a commit that referenced this pull request Oct 29, 2026
Twelve pull requests: #1364, #1409, #1410, #1412, #1413, #1414, #1415, #1416,
#1417, #1418, #1419, #1420.

Test suites: 706 passing across 93 suites, up from 640 across 92. laya-cpp
registers 423 ctest tests, up from 213, and its MIN_TESTS floor is raised to match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants