Skip to content

docs(common): render_criterion's docstring must name the separators it passes - #997

Merged
NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix-render-criterion-separators
Oct 7, 2026
Merged

NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix-render-criterion-separators

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

Fixes render_criterion's docstring in laya/common.py and adds a gate in
tests/test_criteria.py that holds it to the separators the implementation actually passes.

Why

The docstring said:

Strings pass through; anything structured (dict, list, number) becomes compact JSON, so a
rubric reads as JSON rather than a Python repr.

laya/common.py:109 is:

return json.dumps(value, ensure_ascii=False, separators=(", ", ": "), default=str)

Compact JSON is separators=(",", ":"). (", ", ": ") is the default, space-padded spelling --
the opposite of what the docstring promises. Two existing tests already assert the space-padded
bytes, so the code is right and the docstring was the only stale surface:

  • tests/test_criteria.py:50 — check("criterion/list -> json", render_criterion(["a", "b"]), '["a", "b"]')
  • tests/test_batch.py:331 — ("dict level", [{"a": 1}], {"0": '{"a": 1}'}), ("list level", [[1, 2]], {"0": "[1, 2]"})

The word "compact" also reads as a promise about prompt bytes, which is the one thing a caller
cannot re-derive: render_criterion output goes into the tokenized prompt, so whether a member
separator costs a token is a question the maintainer will be asked. Naming the separators makes
the claim checkable.

How

Docstring now states both halves of what the code does, and behaviour is untouched:

    Strings pass through; anything structured (dict, list, number) becomes a single-line JSON
    document with the default separators -- `", "` between members, `": "` before a value --
    so a rubric reads as JSON rather than a Python repr. Without this a dict-valued criterion
    crashed `noul` outright and leaked `{'desc': ...}` into `choice` and `score` prompts.

The gate added at the bottom of tests/test_criteria.py:

  1. AST-reads render_criterion's docstring out of laya/common.py (not a copy of the text);
  2. bans "compact" anywhere in it, case-insensitively;
  3. requires it to spell both ", " and ": ", and to keep stating the property "compact" was
    reaching for ("single-line JSON");
  4. reads the separators= tuple out of the same json.dumps call via AST and pins it to
    (", ", ": "), so the doc and the code are held to each other rather than to this test file;
  5. live-checks the bytes: render_criterion({"a": 1, "b": 2}) == '{"a": 1, "b": 2}', that the
    result differs from json.dumps(..., separators=(",", ":")), that it stays on one line, and
    render_criterion(["a", "b", "c"]) == '["a", "b", "c"]'.

Verification

  • Baseline suite: 198 passed, 0 failed — measured in a separate detached worktree at a4a8921,
    removed after the run, so the number is the base commit's own count.
  • Extended suite: 208 passed, 0 failed (+10 checks).
  • ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120 → All checks passed; same
    selection on tests/test_criteria.py → All checks passed.
  • python -m compileall -q laya/ tests/ → silent.
  • Mutation harness laya-bench/mutate_render_criterion_separators.py → 4/4 CAUGHT, tree restored:
    • m1 pre-fix docstring restored → substring ban fires;
    • m2 "compact" reintroduced alongside the separator sentence ("compact single-line JSON", which
      the first draft's "compact JSON" ban missed) → now caught by the case-insensitive ban;
    • m3 separator spelling dropped from the docstring → both separator-token checks fail;
    • m4 code switched to genuinely compact (",", ":") → the AST separator pin fails, the live byte
      check fails, and the "dict is not compact" witness fails.
  • Secret scan on the full diff (sk-…, ghp_…, AKIA…, PEM header, Bearer …, api_key=…): 0 matches.

Deconfliction

Checklist

  • My PR focuses on one specific issue or feature.
  • I have read and followed CONTRIBUTING.md / AGENTS.md.
  • I have added or updated tests that verify the fix.
  • All CI gates (ruff, compileall, test suites) pass locally.

aashish254 and others added 2 commits October 7, 2026 00:23
…t passes

The docstring promised structured criterion values "become compact JSON". Compact JSON is
separators=(",", ":"); the implementation at laya/common.py:109 passes
separators=(", ", ": "), the default spelling, so a two-member dict renders as
`{"a": 1, "b": 2}` and a list as `["a", "b", "c"]`. tests/test_criteria.py:50 and
tests/test_batch.py:331 already assert the space-padded bytes, so the docstring was the only
surface claiming the opposite.

Reworded to state both halves of what the code really does -- a single-line JSON document with
the default separators -- leaving behaviour untouched. Added a gate to tests/test_criteria.py:
AST-reads render_criterion's docstring, bans "compact" in it at all, requires it to spell both
`", "` and `": "` and to keep the single-line promise, reads the separators tuple out of the
json.dumps call in the same AST rather than retyping it, and live-checks the bytes for a dict
and a list, including that they differ from a genuinely compact dump.

Suite 198 -> 208 (+10 checks; baseline measured in a detached a4a8921 worktree); mutation
harness laya-bench/mutate_render_criterion_separators.py catches 4/4 mutants: the pre-fix
docstring, "compact" reintroduced beside the separators, the separator spelling dropped, and
the code switched to genuinely compact separators.
…the routing/language-detection step -- a native crash on the runner, not an assertion; the same commit is green on the other four matrices)
@NandhaKishorM
NandhaKishorM merged commit ddda1f8 into NandhaKishorM:main Oct 7, 2026
32 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged in 0.3.29.

render_criterion passes specific separators and its docstring did not name them. For a function whose entire job is producing the exact text the model reads, the separators are the contract, and anyone reasoning about option collapse or the head budget needs them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants