Skip to content

fix(integrations): fail closed on malformed nested decision fields - #1335

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
thread3d:fix/integrations-malformed-nested
Oct 24, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
thread3d:fix/integrations-malformed-nested

Conversation

@thread3d

Copy link
Copy Markdown
Contributor

What

Shared checks in laya/integrations/_guard.py (answers_map, answer_number, probability_map, malformed) are applied in CrewAI's and Agno's route() and screen() and in all three LlamaIndex consumers, so answers, each answer, choice, noul and probabilities are validated before use.

Why

#1301 refuses a top-level non-object body and #1321 refuses a non-object answers; one level below both, the consumers still trusted every nested value. They read res.get("answers", {}) (whose {} only fires when the key is ABSENT), iterated it with .items(), used each answer as a dict, called .startswith on choice and compared noul with >=, so a valid object with answers: [], {"q": null}, {"noul": "high"} or {"noul": true} raised AttributeError/TypeError out of the one path whose job is to fail closed. The bool case was worse than a crash: bool subclasses int, so noul: true read as 1.0 and raised a guardrail violation on a body nobody could read; in the LlamaIndex multi-selector a probabilities value of true ranked first as 1.0.

How it was verified

python tests/test_crewai.py
python tests/test_agno.py
python tests/test_llamaindex.py

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main (branch cut from upstream/main at 0.4.14)
  • Tests pass locally
  • Docs or examples updated when the public API changed

NandhaKishorM#1301 refuses a top-level non-object body and NandhaKishorM#1321 refuses a non-object 'answers'. One
level below both, the four consumers still trusted every nested value: they read
res.get("answers", {}) -- whose {} default only fires when the key is ABSENT -- then
iterated it with .items(), used each answer as a dict, called .startswith on 'choice' and
compared 'noul' with '>='. A body that is a valid object but whose answers is a list, or
whose answer is a string/null, or whose noul is a string/null/bool, raised
AttributeError/TypeError out of the one path whose job is to fail closed. The bool case
was worse than a crash: bool subclasses int, so 'noul': true read as 1.0 and raised a
guardrail violation on a body nobody could read, while 'noul': null crashed; on the
llamaindex multi-selector a 'probabilities' value of true ranked first as 1.0.

Shared checks now live in integrations/_guard.py (answers_map, answer_number,
probability_map, malformed) and are applied in crewai's and agno's route() and screen()
and in all three llamaindex consumers. The policy matches the modules' existing one: a
body this code cannot read raises the named RuntimeError ("Malformed decision
response ..."), it is not silently treated as safe. score_violation_probability keeps
its documented degrade-to-0.0 behaviour, since a score distribution is advisory there.

The three registered suites now drive both surfaces of each adapter with six nested
shapes plus a positive control; before the fix they reported the TypeError /
bool-as-1.0 / TypeError failures above.
@NandhaKishorM
NandhaKishorM merged commit 50998fe into NandhaKishorM:main Oct 24, 2026
48 of 52 checks passed
NandhaKishorM added a commit that referenced this pull request Oct 24, 2026
33 PRs. Server hardening (#1331, #1332, #1333), TypeScript parity
(#1330, #1336, #1337, #1338, #1339, #1340, #1341, #1342, #1343),
fail-closed input handling in evals and the integrations (#1317, #1318,
#1319, #1320, #1321, #1322, #1326, #1335), exact and faster script
detection (#1334, #1349), JVM and .NET resource handling (#1327, #1328,
#1329, #1346), port goldens and clipping (#1324 excluded, #1325, #1344,
#1345), CI (#1323, #1347, #1348).

Verified before tagging: all 92 suites pass, 610 tests. ruff clean on
both lists. laya-ts 1247 tests across 44 files, tsc --noEmit clean.
Version agrees across pyproject.toml, laya.__version__ and nix/package.nix.
README carries 0.4.15 only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@thread3d
thread3d deleted the fix/integrations-malformed-nested branch October 28, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants