Skip to content

docs(examples): example 26 must count triage_questions()'s three yes/no fields - #978

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:fix-example26-preset-count
Oct 7, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:fix-example26-preset-count

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

examples/26_presets_triage.py opens its banner with a primitive count that the schema it names
does not return. The banner says:

triage_questions() is a ready-made schema for inbound support: five questions, one
forward pass. It mixes all three primitives -- a multi-way choice, two yes/no
probabilities
, and an ordinal score -- so a single call fills a whole triage record.

But laya/presets.py::triage_questions() ships three noul fields -- is_urgent,
refund_requested, churn_risk -- plus one choice (intent) and one score
(frustration). The example's own print block already prints all three of them
(examples/26_presets_triage.py:40-45), so the banner disagreed with the page underneath it.
The banner's own arithmetic is 1 + 2 + 1 = 4 against its stated "five questions".

One word changes: "two" -> "three". A gate is added to tests/test_confidence.py so the
banner's stated counts stay pinned to what triage_questions() returns.

Why

The count is not a stylistic thing to get wrong:

Gate design

tests/test_confidence.py gains a fifth gate section (# ----- example 26's banner must count the primitives triage_questions() returns). Three rules:

  1. test_triage_preset_is_one_choice_three_noul_one_score -- the anchor. Calls
    laya.presets.triage_questions() directly (no weights) and asserts the returned dict
    is 5 keys with the exact type mix {choice:1, noul:3, score:1}. If the schema ever moves,
    this breaks first and the banner test below follows -- the drift is loud, not silent.
  2. test_example_26_banner_counts_match_the_preset -- reads the shipped example source, extracts
    each stated number with a one|two|three|four|five|six regex, and requires every one to
    match the schema. Also asserts the arithmetic inside the same sentence adds up:
    choice + noul + score == total.
  3. test_example_26_drops_the_wrong_yes_no_count -- the ban. A regex \btwo\s+yes/no\s+probabilit
    is verified to fire on an inline OLD_BANNER_26 literal of the pre-fix wording and to not
    fire on the shipped example. The same positive regexes are asserted to read OLD_BANNER_26
    as the pre-fix "two", so the positive rule was genuinely falsifiable -- no ban is a guess.

The gate is pure-Python: no weights, no device, no network.

Verification

$ python tests/test_confidence.py
137 passed, 0 failed
all confidence tests passed

$ ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
All checks passed!

$ ruff check tests/test_confidence.py --select=E9,F63,F7,F82,F401,F811 --line-length=120
All checks passed!

$ python -m compileall -q laya/ tests/

Revert-the-fix witness (change only three back to two on disk, re-run):

135 passed, 2 failed
  FAIL page-26/test_example_26_banner_counts_match_the_preset: banner says 'two' yes/no probabilities, triage_questions() returns 3 noul fields
  FAIL page-26/test_example_26_drops_the_wrong_yes_no_count: the wrong yes/no count is back in the shipped example

Mutation harness at laya-bench/mutate_example26_count.py, seven variants, each byte-restored,
run with PYTHONDONTWRITEBYTECODE=1:

baseline: OK (all confidence tests passed)

m1 banner says two yes/no                                       CAUGHT
m2 banner says one yes/no (arithmetic 1+1+1=3)                  CAUGHT
m3 banner says four yes/no (arithmetic 1+4+1=6)                 CAUGHT
m4 banner says six questions                                    CAUGHT
m5 banner says four questions                                   CAUGHT
m6 drop one noul field from the schema                          CAUGHT
m7 turn churn_risk into a score, banner still says three noul   CAUGHT

7/7 caught

Mutants m6 and m7 change laya/presets.py itself, not just the prose. The schema anchor
test_triage_preset_is_one_choice_three_noul_one_score fails on m6 and the banner-count
positive rule fails on both, so the gate reads the actual dict, not a table.

Relationship to prior work

Same class as the example-page gates this quarter (#844, #850, #972, #973, #974, #975) and the
page-28 gate already living in this same file: the codebase asserts a fact the code does not
verify. This PR is the primitive-count version.

Nothing about the schema, the runtime, or the model changes. The only code edit is one word in
one banner.

Checklist

  • One coherent change: the banner's yes/no count and the gate that pins it.
  • Rebased on latest main (a4a8921 release: 0.3.28).
  • python tests/test_confidence.py -- 137/0 green.
  • ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120 clean.
  • python -m compileall -q laya/ tests/ clean.
  • No public API change, so tests/test_hooks_api.py is untouched.
  • English-only, conventional commit prefixes, no wholesale reformat.
  • Diff contains no secrets, keys, tokens, or credentials.

…rimitives

The example opened with "five questions ... a multi-way choice, two yes/no
probabilities, and an ordinal score". laya.presets.triage_questions() ships
three noul fields -- is_urgent, refund_requested, churn_risk -- plus the one
choice and the one score, so the banner's own arithmetic was 1 + 2 + 1 = 4
against its stated "five questions". Fix the count to "three" and add a gate
that reads the banner's stated numbers and the schema's actual type mix and
refuses to pass when they disagree.
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged in 0.3.29.

triage_questions() returns three yes/no fields and the page counted otherwise, which is the kind of error that survives indefinitely because nobody counts. Reading the count out of the preset itself means the page follows the preset the next time a field is added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants