Skip to content

fix(calibrate): a falsy non-object temperature_by_options is a refused shape - #1109

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:fix-calibrate-falsy-temperature-by-options
Oct 11, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:fix-calibrate-falsy-temperature-by-options

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

apply_calibration_payload now refuses a temperature_by_options that is a falsy non-object.

by_options = payload.get("temperature_by_options") or {}
if not isinstance(by_options, dict):
    raise ValueError(...)

becomes

by_options = payload.get("temperature_by_options")
if by_options is None:   # absent, or an explicit null: a file from before bucket temperatures
    by_options = {}
if not isinstance(by_options, dict):
    raise ValueError(...)

tests/test_calibrate.py gains seven checks: five refusals and two that pin the tolerant half
(null and an absent key still install an empty map).

Why

The or {} swallows JSON's 0, 0.0, "", false and [] before the type check ever sees them,
so those payloads load "successfully" with temperature_by_options == {}. The user-visible result is
a calibration file whose per-bucket temperatures are gone — every bucket answers at the per-type
temperature — with no error and no warning.

Three things make this a defect rather than a policy:

  1. apply_calibration_payload's own docstring lists it among the refused shapes: "a
    temperature_by_options that is not an object, a binning_map that is not an object ... is a
    file this code cannot read." The two fields named in the same sentence behave oppositely —
    binning_map is tested with is not None, so it refuses all five of these values.
  2. The docstring's closing example is exactly this failure class: a shape that "used to pass the
    length check and install its keys as temperatures."
  3. The truthy cases are already pinned (test_calibrate.py:645-649 refuses the scalar 5 and a
    non-empty list of pairs [["choice:2", 1.5]]), so the check exists and works — it is only the
    falsy values that slip through it. A file truncated or mis-edited into an empty container is the
    case that most needs the error, and it is the case that currently produces none.

null is deliberately not made a refused shape: a payload written before bucket temperatures
existed simply has no key, and an explicit null means the same thing. Keeping that half is why the
guard is is None rather than dropping the default.

No tracked issue for this — it came out of a docstring-vs-behavior audit of laya/calibrate.py, and
I could not find it among the open PRs: laya/calibrate.py is touched by zero of them.

How it was verified

python tests/test_calibrate.py
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
python -m compileall -q laya/ tests/
python scripts/test_suites.py
  • Reproduction, before the change (each line is a real ValueError-or-load result printed by
    calling apply_calibration_payload directly):
    tbo=[] -> LOADED tbo={}, tbo=0 -> LOADED tbo={}, tbo='' -> LOADED tbo={},
    tbo=False -> LOADED tbo={}, while
    tbo=5 -> ValueError: ... must be an object of bucket -> float, got int, and, for the sibling
    field, bm=[] -> ValueError: calibration JSON binning_map must be an object ... — the asymmetry
    between the two fields the same docstring sentence lists. (0.0 is covered by the new checks
    rather than by this probe; see the next bullet, where it fails on the unfixed code.)
  • python tests/test_calibrate.py: 240 passed, 0 failed on this branch (2.90 s), against
    233 passed, 0 failed on the pre-fix code (3.35 s) — the same file plus the seven new checks, so
    the runtime is unchanged. The pre-fix arm ran in a tree whose laya/calibrate.py and
    tests/test_calibrate.py are byte-identical to upstream/main (confirmed with
    git diff --stat upstream/main -- laya/calibrate.py tests/test_calibrate.py, which prints nothing).
  • The new tests against the unfixed laya/calibrate.py (fix reverted from the same tree, tests
    kept): 235 passed, 5 failed, and the five failures are exactly the new
    shape/falsy non-object by_options 0 / 0.0 / '' / False / [] checks. No pre-existing check
    changed state.
  • Over-broad-fix check: replacing the guard so an explicit null is refused (while an absent key is
    tolerated) dies at tests/test_calibrate.py:668, the new
    shape/null by_options installs an empty map call — so that check is load-bearing, not decoration.
  • ruff check laya/ ... → All checks passed!; python -m compileall -q laya/ tests/ → exit 0.
  • python scripts/test_suites.py → all 86 suites passed (469 passed, 1 skipped), identical to
    the numbers on main, so nothing that passes a bucket map through the loader broke.
  • Diff scanned for keys/tokens: 0 hits.
  • I checked the docs prose that mentions temperature_by_options (docs/finetune.md,
    README.md, the three integration pages); none of them states the falsy-tolerance, so no page goes
    stale with this change.

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main
  • Tests pass locally
  • Docs or examples updated when the public API changed

…d shape

`payload.get("temperature_by_options") or {}` hid JSON's `0`, `0.0`, `""`,
`false` and `[]` behind an empty bucket map, so a file this code cannot read
loaded and silently dropped every per-bucket temperature. The docstring names
that field as refused, and its sibling `binning_map` already refuses the same
values. `null` and an absent key stay tolerant -- that is the file-from-before
case -- so the guard is `is None`, matching `binning_map`'s policy.
@NandhaKishorM
NandhaKishorM merged commit 5a6760c into NandhaKishorM:main Oct 11, 2026
38 of 40 checks passed
NandhaKishorM added a commit that referenced this pull request Oct 11, 2026
35 pull requests from 11 contributors. Headline #1072: sdk/typescript's
response validation rejected the x_jev_confidence that 0.4.2's #1066 started
sending, so the TypeScript SDK lane has been red on main since that release.

Also #1073 (LAYA_TOURNAMENT_GROUP), #1083 (laya.jev_confidence), #1105
(model: null across all four MCP tools), #1104, #1109, #1112, #1113,
#1108 and #1117 (laya-ts), #1090 (Java floors remeasured on the merged
tree), and 20 documentation and test-gate corrections.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged in 0.4.3. payload.get(...) or {} swallowing the falsy values before the type check ever runs is a precise bug report, and the fix is the right three lines.

What I measured on the merged tree, calling apply_calibration_payload with a valid temperature and each falsy non-object in turn:

  • 0 -> ValueError: calibration JSON temperature_by_options must be an object of bucket -> float, got int
  • 0.0 -> same, got float
  • '' -> same, got str
  • False -> same, got bool
  • [] -> same, got list
  • None -> accepted, installs an empty map

So all five previously-silent shapes are refused and the tolerant half is intact. Keeping None tolerant is the important half of the distinction: a file written before bucket temperatures existed has no business failing to load, and an explicit null means the same thing. A 0 in that field means the writer thought they were setting something.

A calibration file that loads "successfully" with an empty temperature map is the worst outcome of the three, because confidence then comes back uncalibrated with no signal that the file was misread. Verified: tests/test_calibrate.py passes with your seven added checks, all 87 suites pass, and the 94-row real-weights decision diff is identical to 68804629.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants