Repository navigation
fix(dotnet): parse and re-emit JSON without recursion, so depth cannot overflow the stack - #1346
Merged
NandhaKishorM merged 1 commit intoOct 24, 2026
Conversation
…t overflow the stack
MaxDepth mirrors CPython (2997 levels, which Python 3.12 loads) and is checked before
every descent, but reaching it costs about two managed frames per JSON level: on a thread
with the platform-default 1 MB stack the stack runs out at roughly half that depth, and
.NET cannot catch a stack overflow. So a document CPython accepts did not produce an error
here -- it aborted the process. The port's own suite demonstrated it: running
PythonJsonErrorTests killed the test host with "Stack overflow. Repeat N times: at
State.Array/State.Value" and exit 134, which is why those tests could only ever pass on a
runner that happened to hand out a larger stack.
Three walks recursed over the same depth, so all three now carry an explicit stack:
* State.Value/Object/Array -> one loop with a Stack<Frame> of open containers. The depth
limit reads the open-container count, which is the number the old `depth` parameter
carried, so 2997 levels still load and 2998 still raise. Tokens, scalar types, error
texts and their exact positions, last-value-wins duplicate keys and the "{} / []"
fast paths are unchanged.
* Write (the re-emission ParseDocument performs for tokenizer_config.json) -> a
List<WriteFrame> of container enumerators; Utf8JsonWriter already keeps its own depth
counter and does not recurse.
* IsAlteredByReEmission -> the same explicit-stack walk.
Verified by building the port for net8.0 in a scratch copy (it pins SDK 10.0.100, which
this machine does not have; local-only shims for the .NET 9 APIs it uses):
* the whole suite now runs to completion: 2670 passed, 0 failed, 752 skipped (the ONNX
artifact classes) -- the same 85 tests that used to abort PythonJsonErrorTests included;
* the 85 pinned Python error texts, 2997-loads / 2998-rejected and the re-emission at
10/65/900/2990 levels all pass;
* new test: parse + re-emit of a 2990-level document on a 256 KB thread, which aborted
the host before this change and now passes, so the property is the fix rather than the
runner's stack;
* a local differential check (2000 generated documents) compared every parsed and
re-emitted tree against System.Text.Json structurally: identical kinds, values, member
order and strings. Number spelling is deliberately not compared -- laya follows
Python's spellings (-1.25e-7 re-emits as -1.25E-07 through Utf8JsonWriter), which is
pre-existing behaviour and unchanged here.
thread3d
added a commit
to thread3d/laya
that referenced
this pull request
Oct 24, 2026
The fork's own release: upstream 0.4.14 plus this fork's patch set and the seventeen fixes verified in this checkout (each proposed upstream as its own PR, NandhaKishorM#1331-NandhaKishorM#1346). Declared in the three places tests/test_packaging.py holds equal - pyproject.toml, laya/__init__.py and nix/package.nix - and in the routing golden meta, which records laya.__version__ and regenerates byte-identically at this commit. README's release notes rotate to 0.4.15. Gates: canonical suite 89 invocations with only the documented langchain-extra failure, ruff/compileall, packaging (173 checks), doc tables (181), env docs (67), citations (22); TypeScript 1237 vitest tests, TS client 32, Rust 27 integration targets, .NET whole suite 2670 on a net8 retarget (the port pins SDK 10.0.100, absent here). The sdist and wheel built from this commit install and report 0.4.15.
NandhaKishorM
added a commit
that referenced
this pull request
Oct 24, 2026
33 PRs. Server hardening (#1331, #1332, #1333), TypeScript parity (#1330, #1336, #1337, #1338, #1339, #1340, #1341, #1342, #1343), fail-closed input handling in evals and the integrations (#1317, #1318, #1319, #1320, #1321, #1322, #1326, #1335), exact and faster script detection (#1334, #1349), JVM and .NET resource handling (#1327, #1328, #1329, #1346), port goldens and clipping (#1324 excluded, #1325, #1344, #1345), CI (#1323, #1347, #1348). Verified before tagging: all 92 suites pass, 610 tests. ruff clean on both lists. laya-ts 1247 tests across 44 files, tsc --noEmit clean. Version agrees across pyproject.toml, laya.__version__ and nix/package.nix. README carries 0.4.15 only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
laya-dotnet/src/Laya/PythonJsonReader.csparses, re-emits and walks JSON without recursion: the parser keeps open containers in aStack<Frame>, the re-emitter aList<WriteFrame>of container enumerators, andIsAlteredByReEmissionits own explicit stack. One new test pins the property on a 256 KB thread.Why
MaxDepthmirrors CPython (2997 levels, which Python 3.12 loads) and is checked before every descent, but reaching it costs about two managed frames per JSON level. On a thread with the platform-default 1 MB stack the stack is exhausted at roughly half that depth, and .NET cannot catch a stack overflow — so a document CPython accepts did not produce a catchable error here, it aborted the process. The port's own suite showed it: runningPythonJsonErrorTestskilled the test host withwhich is why those tests could only ever pass on a runner that happened to hand out a larger stack. A downloaded
tokenizer_config.jsonis exactly the untrusted document this parses.How it was verified
Built for
net8.0in a scratch copy, because the project pins SDK 10.0.100 /net10.0, which this machine does not have (local-only shims for the three .NET 9 APIs it uses; nothing shimmed is part of the patch):PythonJsonErrorTeststhat used to abort it (before: exit 134).DeepNestingNeedsNoStackProportionalToItsDepthparses and re-emits a 2990-level document on a 256 KB thread; it aborted the host before this change and passes now, so the fix is what is being tested rather than the runner's stack.System.Text.Jsonstructurally — identical kinds, values, member order and strings. Number spelling is deliberately not compared:layafollows Python's spellings (-1.25e-7re-emits as-1.25E-07throughUtf8JsonWriter), which is pre-existing and unchanged.No workflow change is needed:
dotnet.ymlfilters only the model-backed classes, soPythonJsonErrorTestsis already in the lane — it just could not survive it.Checklist
main(branch cut fromupstream/mainat 0.4.14)