Repository navigation
docs(guardrails): the #781 page and suite must report only what #781 measured - #1084
Conversation
…ly what NandhaKishorM#781 measured NandhaKishorM#1032 published `docs/guardrail-limitations.md` and `tests/test_guard_document_781.py` carrying numbers issue NandhaKishorM#781 never produced, and it stated the reporter's measurements as repository benchmarks with no attribution. Removed, because nothing measures them: - `# ~0.85+ (detected)` and `# ~0.18 (may pass!)` printed next to a two-call example. The real c011 figure is 0.182; 0.85 was never run. - A three-point "Why This Happens" list, asserted as mechanism. NandhaKishorM#781 reports where the head under-fires, not why. - `process_document()`, which references an undefined `r`, `q` and `GuardViolation`, so it cannot run, plus a 0.3 intermediate threshold nothing validates. - A "Future Work" roadmap and a threshold-mode section (0.3 / 0.5 / 0.7) that NandhaKishorM#781's own threshold sweep contradicts: lowering the cut recovers only c012, and at 0.15 nine of the reporter's 28 should-allow cases are already blocked. - `python -m pytest tests/test_guard_document_781.py -v`. There is no pytest config in this repo, so that command collects nothing; the suite runs through `scripts/test_suites.py`. Added, all of it from NandhaKishorM#781: the c011/c012/c042 score table, the c043 matched control, the c038 inversion, the three recall/FPR rates at 0.5, the pinned revision and version setup, and the reporter's own reproduction snippet. Every figure is attributed to the reporter's 53-case set on the reporter's machine. The suite's docstring claimed ">90% recall at 0.5 threshold" for direct attacks. NandhaKishorM#781 reports ten jailbreak strings at `jailbreak >= 0.985` and a set-wide recall of 0.72, so the figure is neither of those. The printed note now names the set the rates come from and says this file does not measure them, and the case list no longer describes the two direct-attack strings as NandhaKishorM#781 reproduction data. No test case, threshold or assertion changed: `passed_direct >= 2` and the five inputs are byte-identical, and the suite still prints 3/3.
|
Merged in 0.4.3. Deleting numbers nobody ran is worth more than the lines it removes, and The Verified on the merged tree: Net 95 added against 140 removed on a page about a known limitation is a good trade. |
What
Replaces the numbers in
docs/guardrail-limitations.mdandtests/test_guard_document_781.pythat issue #781 never produced with the ones it did, and attributes every figure to its reporter.
Removed, because nothing in the repo or in #781 measures them:
# ~0.85+ (detected)and# ~0.18 (may pass!)printed beside a two-call example. [bug]:guard:prompt_injectionscores 0.00 to 0.30 on instructions embedded in document-shaped text (3 cases, with a matched control) #781's realdocument-shaped scores are 0.182, 0.301 and 0.000; 0.85 was never run.
guard:prompt_injectionscores 0.00 to 0.30 on instructions embedded in document-shaped text (3 cases, with a matched control) #781 reports where the head under-fires,not why.
process_document(), which references an undefinedr,qandGuardViolationso it cannot run,together with a 0.3 intermediate threshold nothing validates.
guard:prompt_injectionscores 0.00 to 0.30 on instructions embedded in document-shaped text (3 cases, with a matched control) #781's own thresholdsweep contradicts: lowering the cut recovers only c012, and at 0.15 nine of the reporter's 28
should-allow cases are already blocked.
python -m pytest tests/test_guard_document_781.py -v. There is no pytest configuration in thisrepo, so that command collects nothing; the suite runs through
scripts/test_suites.py, whichlists it at line 89.
guard:prompt_injectionscores 0.00 to 0.30 on instructions embedded in document-shaped text (3 cases, with a matched control) #781 reports tenjailbreak strings at
jailbreak >= 0.985and a set-wide recall of 0.72 — the claim is neither.The printed note now names the set its rates come from and states that this file does not measure
them, and the case-list comment no longer calls the two direct-attack strings "[bug]:
guard:prompt_injectionscores 0.00 to 0.30 on instructions embedded in document-shaped text (3 cases, with a matched control) #781 publicreproduction data".
Added, all of it taken from #781: the c011 / c012 / c042 score table, the c043 matched control, the
c038 inversion, the three recall/FPR rates at threshold 0.5, the pinned revision and version setup,
and the reporter's own reproduction snippet.
No test case, threshold or assertion changed. The five inputs, the 0.5 comparison and
assert passed_direct >= 2are byte-identical; only prose moved. Page size 139 → 84 lines.Why
#1032 published a docs page whose example outputs and mitigation code were written rather than
measured, and presented a third party's benchmark as repository benchmarks with no attribution. A
deployer reading it would believe the guard scores ~0.85 on a direct attack (never verified here),
that document section-splitting is a validated mitigation (it is not), and that
pytestruns thesuite (it does not collect it). The suite is wired into
scripts/test_suites.py, so its docstringis the next thing a reader takes as fact.
This is repair work on my own merged PR, not a claim on #781: the scope question there — whether
indirect, embedded injection belongs in the guard presets' coverage — is still open, and
#892 is the fixture-side work. Nothing here changes guard behaviour, so nothing here closes an
issue.
How it was verified
Every retained number was re-read against the #781 body rather than transcribed from the previous
version of the page: 53 cases (25 should-block, 28 should-allow), c011 0.000/0.182/0.82, c012
0.005/0.301/0.78, c042 0.000/0.000/0.51, c043 control at 0.000, c038 inversion at 0.619, and
0.72/0.21, 0.60/0.18, 0.76/0.29 at threshold 0.5 on revision
55cf4c4ebb4ebe31b2550e8bdf3bd21b99753851.Checklist
mainFixes # — deliberately none: this corrects the record about #781 and changes no behaviour, so it
should not close it.