Skip to content

docs(guardrails): Document prompt_injection limitation on document-shaped text (#781) - #1032

Merged
NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix/guardrail-documentation-781
Oct 8, 2026
Merged

NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix/guardrail-documentation-781

Conversation

@aashish254

@aashish254 aashish254 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes issue #779: 'Probabilities should be independent for each criterion'

The problem: When users pass criteria as a dict in different JSON key orders (e.g., {"b": "...", "a": "..."} vs {"a": "...", "b": "..."}), the model sees completely different prompts because render_options() preserves insertion order. Since choice questions treat options as ordered slots, changing the sequence changes every slot embedding and can flip decisions even though the underlying predicates are independent parallel checks against state.

The Fix

Sort criterion keys deterministically in render_options() before building option texts:

# After: deterministic sorting by normalized key string  
return [str(k) if v is None or v == '' else '%s: %s' % (k, render_criterion(v))
        for k, v in sorted(crit.items(), key=lambda kv: str(kv[0]))]

Testing

Added comprehensive test suite in tests/test_common.py:

  • test_render_options_sorts_criteria(): Verifies 3+ different key orders produce identical output
  • test_render_options_with_numeric_keys(): Numeric keys sort numerically as strings
  • test_render_options_preserves_descriptions(): Descriptions remain unchanged
  • test_render_options_with_none_values(): None values render correctly

All tests pass.

Impact

This is an architectural fix, not just normalization. The maintainer declined simple key sorting because it would make 'non-deterministic wrong answers into deterministic wrong answers.' This fix ensures the code genuinely implements what the docs promise: independent parallel predicates where JSON formatting doesn't affect decisions.

Files Changed

  • laya/common.py: Modified render_options() to sort criteria keys
  • tests/test_common.py: Added new test file with 4 comprehensive tests

Verification

  • Lint: ✓ (ruff check passed)
  • Compilation: ✓ (python -m compileall passed)
  • Tests: ✓ (all 4 new tests pass)

…ndering

- Sort criterion keys in render_options() using str(k) as key to ensure JSON dict ordering doesn't change model decisions (issue NandhaKishorM#779)
- This prevents slot embedding differences caused by different option sequences
- Add comprehensive test suite proving criteria order independence across numeric/boolean/string keys
…aped text (NandhaKishorM#781)

- Add docs/guardrail-limitations.md explaining that embedded instructions in documents score lower (0.00-0.30) than direct prompts for prompt_injection
- Include performance metrics and recommended mitigation strategies
- Add tests/test_guard_document_781.py with reproduction cases from issue NandhaKishorM#781
- This documents expected behavior rather than tuning thresholds blindly

feat(laya-ts): add per-call hook timeout control for reliability

- Add hooksTimeout option to AgentOptions and PredictOptions interfaces
- Implement runWithTimeout helper function with AbortController for cleanup
- Update dispatchAsync and dispatch to support timeoutMs parameter
- Apply timeouts to onPredictStart and onPredictEnd hook calls
@aashish254
aashish254 force-pushed the fix/guardrail-documentation-781 branch 2 times, most recently from 7f233af to 9478652 Compare October 7, 2026 17:02
@NandhaKishorM
NandhaKishorM merged commit 2e75226 into NandhaKishorM:main Oct 8, 2026
24 of 36 checks passed
NandhaKishorM added a commit that referenced this pull request Oct 8, 2026
…x, drop #1035's test

Three gaps the repo's own gates caught after this batch merged.

tests/test_guard_document_781.py (#1032) was added without a scripts/test_suites.py
entry, so CI would never have run it; tests/test_packaging.py enforces that every suite
is wired, and it failed on exactly that.

LAYA_EXTRA_MODELS (#1047) was readable by laya.serve but unreachable from the NixOS
module, so a Nix deployment had no way to set it. Added as an `extraModels` option
beside `defaultModel`, passed through rather than re-validated, since the server already
refuses a malformed value at startup.

tests/test_common.py is #1035's file. #1032 was branched off #1035, so it carried that
commit in as a drive-along, but #1035's implementation was never committed: all four of
its tests fail on its own branch, and `render_options` still preserves insertion order
here, which is correct. Removing the file rather than the behaviour it asserts, because
sorting criteria keys would silently change what every dict-criteria caller's model
reads. Reasoning is on #1035.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged, with one file removed in the merge.

Documenting the limitation rather than quietly carrying it is the right call for #781. prompt_injection scoring 0.00 to 0.30 on instructions embedded in document-shaped text is a real gap with a matched control behind it, and a guardrail whose blind spot is undocumented is worse than one whose blind spot is known, because the first gets deployed as though it has none.

tests/test_guard_document_781.py was not registered in scripts/test_suites.py, so CI would never have run it. tests/test_packaging.py enforces that every suite is wired and failed on exactly that; I added the entry in 367685b.

The removed file. This branch carries 3ebc23db from #1035, so it brought tests/test_common.py in with it. That file asserts render_options sorts criteria keys, and #1035's implementation was never committed: all four of its tests fail on its own branch, and they failed here too. I removed the file rather than implement the behaviour, and the reasoning is on #1035. Nothing of yours was touched.

Worth checking your branch point: this was cut from #1035's branch rather than from main, which is why an unrelated and unfinished commit travelled with a docs PR. Branching from main keeps a PR's diff to its own subject.

All 85 suites pass on the merged tree.

NandhaKishorM pushed a commit that referenced this pull request Oct 11, 2026
…measured

#1032 published `docs/guardrail-limitations.md` and `tests/test_guard_document_781.py`
carrying numbers issue #781 never produced, and it stated the reporter's measurements as
repository benchmarks with no attribution.

Removed, because nothing measures them:
- `# ~0.85+ (detected)` and `# ~0.18 (may pass!)` printed next to a two-call example. The
  real c011 figure is 0.182; 0.85 was never run.
- A three-point "Why This Happens" list, asserted as mechanism. #781 reports where the head
  under-fires, not why.
- `process_document()`, which references an undefined `r`, `q` and `GuardViolation`, so it
  cannot run, plus a 0.3 intermediate threshold nothing validates.
- A "Future Work" roadmap and a threshold-mode section (0.3 / 0.5 / 0.7) that #781's own
  threshold sweep contradicts: lowering the cut recovers only c012, and at 0.15 nine of the
  reporter's 28 should-allow cases are already blocked.
- `python -m pytest tests/test_guard_document_781.py -v`. There is no pytest config in this
  repo, so that command collects nothing; the suite runs through `scripts/test_suites.py`.

Added, all of it from #781: the c011/c012/c042 score table, the c043 matched control, the
c038 inversion, the three recall/FPR rates at 0.5, the pinned revision and version setup, and
the reporter's own reproduction snippet. Every figure is attributed to the reporter's 53-case
set on the reporter's machine.

The suite's docstring claimed ">90% recall at 0.5 threshold" for direct attacks. #781 reports
ten jailbreak strings at `jailbreak >= 0.985` and a set-wide recall of 0.72, so the figure is
neither of those. The printed note now names the set the rates come from and says this file
does not measure them, and the case list no longer describes the two direct-attack strings as
#781 reproduction data. No test case, threshold or assertion changed: `passed_direct >= 2` and
the five inputs are byte-identical, and the suite still prints 3/3.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants