Skip to content

fix(serve): refuse an unpaired surrogate on /v1/systemone/batch too - #814

Merged
NandhaKishorM merged 3 commits into
NandhaKishorM:mainfrom
aashish254:fix-serve-batch-surrogate-v2
Oct 1, 2026
Merged

NandhaKishorM merged 3 commits into
NandhaKishorM:mainfrom
aashish254:fix-serve-batch-surrogate-v2

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

/v1/systemone/batch had no lone-surrogate guard while /v1/systemone has had one since 06462bf,
even though it tokenizes the same body. This puts the same check on the batch route, lifts the
400 message into a module constant both handlers raise with so the two cannot drift, and pins
both routes against the same strings in tests/test_serve.py.

Three files: laya/serve.py (the guard in _systemone_batch_inner, the shared
_LONE_SURROGATE_DETAIL, and the module docstring, which named only one of the two routes),
tests/test_serve.py (8 new cases), docs/http-api.md (the 400 row and a paragraph on why that
status looks unusual, plus the page's "Not (yet) here" section, which still denied a batch endpoint
that has shipped for releases).

Why

06462bf turned a lone \udXXX escape into a 400 on the single route and recorded what it had not
covered -- "only /v1/systemone is checked". The batch route went on walking no guard. Measured on
4185647 with the cached english checkpoint on CPU:

slot              Router.predict    Router.predict_batch  HTTP /v1/systemone  HTTP /v1/systemone/batch
state             TypeError         TypeError             400                 200
instructions      TypeError         TypeError             400                 200
criteria label    TypeError         TypeError             400                 200
paired emoji      ok (1 answers)    ok (1 answers)       200                 200

The TypeError is TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]], raised from laya/agent.py:1028 _encode_state() through laya/common.py:35 encode_text(). The HTTP columns were measured with a stub router that does not tokenize -- with a
real one all three batch bodies are 500 inference failed, because this route maps every
except Exception to that string and writes a _log.exception traceback per request. So a
malformed string in the caller's own body was reported as a server fault to the caller and as an
operator-facing traceback to the log, which is exactly the shape 06462bf exists to stop; the
difference is only that on this route a batch carries several states, so an escape in the second
one was enough to get there.

The guard walks the whole parsed body, not states, because predict_batch tokenizes every state
and every question -- the instructions and criteria-label slots are the same bug through the
other field. It is ordered as on the single route: after _check_batch_limits and
_refuse_body_refusals, so MAX_BATCH_STATES, MAX_STATE_CHARS and MAX_QUESTIONS bound what
the walk can reach. A paired surrogate is one ordinary astral character by the time json.loads is
done, so an emoji keeps reaching the router -- asserted for both routes, since a check keyed on
surrogate code points rather than unpairedness would reject those too.

How it was verified

python -m pytest tests/test_serve.py -q   # 184 passed on main -> 192 passed on this branch
python tests/test_packaging.py           # 126 passed, 0 failed
python tests/test_env_docs.py            # exit 0
python tests/test_doc_tables.py          # exit 0
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120   # All checks passed!
python -m compileall -q laya/ tests/     # exit 0

The 8 new cases are 3 slots x 2 routes for the refusal, plus the paired-surrogate arm per route.
Each refusal asserts the 400, the message read from laya.serve rather than retyped in the test,
and that the router recorded neither a predict nor a predict_batch call -- so a guard placed
after the forward pass fails even though it still returns a 400.

Mutating the branch is what shows the tests bite, and which one shows each half is load-bearing. 14
arms, each a fresh copy of the tree, run against the whole serve suite; an arm passes only when the
failing test ids are exactly the set it should break:

control                       192 passed
A1 batch guard deleted        3 failed  -> the three *-batch refusals
A2 single guard deleted       4 failed  -> the three *-single refusals + 06462bf's test
A3 batch walks states only    2 failed  -> [criteria label-batch], [instructions-batch]
A4 batch walks questions only 1 failed  -> [state-batch]
A5 batch checks states[0]     3 failed  -> the three *-batch refusals
A6 batch answers another msg  3 failed  -> the three *-batch refusals
A7 batch answers 422          3 failed  -> the three *-batch refusals
A8 regex high surrogates only 3 failed  -> both [instructions] arms + 06462bf's test
A9 regex rejects all astral   3 failed  -> both emoji arms + 06462bf's test
A10 guard after forward pass  3 failed  -> the three *-batch refusals
B1 = A1 with the 8 new tests cut out        184 passed  (nothing else in CI notices)
B3 = A10 with the 8 new tests cut out       184 passed
B4 = A3 with the 8 new tests cut out        184 passed

0 of 14 arms misbehaving. The three B arms are the point of this PR: the batch route's guard had
no test at all, so every one of those drifts is invisible to the suite as it stands on main.

No public API changed: both routes accept exactly the bodies they accepted before, and the only
response that differs is that three batch bodies which used to reach inference now answer 400.
tests/test_hooks_api.py is the hook API contract; this touches neither hooks nor that contract.

Related work I can see: #716 and #667 both edit tests/test_serve.py, and #667 edits the
create_app body next to this guard (GET /v1/models); neither adds a surrogate check -- I read
their diffs for surrogate and has_lone_surrogate and both come back empty. #774 merged while
this branch was being built, so it is rebased on top of it: the batch route now shares the single
route's control validation and hook refusals, which is why the docs paragraph here says every
check above applies to both routes rather than carving an exception out for the 422 row.

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main
  • Tests pass locally
  • Docs or examples updated when the public API changed

Fixes # (no open issue: the gap is the limit 06462bf wrote down for itself -- "only /v1/systemone
is checked" -- and no issue proposes carrying it to the batch route)

06462bf turned a lone `\udXXX` escape into a 400 on /v1/systemone and recorded the
limit as "only /v1/systemone is checked". The batch route tokenizes the same body,
so on it the caller's own string still came back as a 500 inference failed.

The detail message moves to a module constant both handlers raise with, so the two
routes cannot drift on what they answer.
Three slots per route, each asserted for the 400, the shared message and the router
never seeing the body, plus a paired-surrogate arm so the guard cannot be widened
into rejecting every emoji.
…e batch route

The page said there was no batch endpoint while /v1/systemone/batch had shipped for
releases; it now names the route and states which of the checks above it shares.
@NandhaKishorM
NandhaKishorM merged commit f5c1e47 into NandhaKishorM:main Oct 1, 2026
22 of 25 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged for 0.3.24.

Lifting the message into _LONE_SURROGATE_DETAIL so both handlers raise with the same string is the part that makes this stay fixed, rather than the guard itself. A second copy of that sentence would have drifted the first time someone reworded one of them.

The docs/http-api.md cleanup is worth its own mention: the "Not (yet) here" section was denying a batch endpoint that has shipped for releases, which is the kind of stale negative claim nobody notices because nobody reads a page looking for things it says do not exist.

Verified on the integrated branch: tests/test_serve.py 195 passed, full test lane green, zero decision flips against 0.3.23 on the real checkpoints.

Thank you.

NandhaKishorM pushed a commit that referenced this pull request Nov 3, 2026
stateText sliced its 4000-character budget with String.slice, which
counts UTF-16 units, so a state whose 4000th unit was the high half of
a surrogate pair returned text ending in an orphaned surrogate --
malformed text Python's state_text never produces, and the same
lone-surrogate shape laya-serve already refuses as a request (#454,
#814). The segment scan and the 4000-character leaf samples cut the
same way. cutCodePoints mirrors the email port's helper, added there
when a budget cut an emoji in half.

Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants