Skip to content

feat(java): laya-java: a JVM inference package, gated against the Python reference - #927

Merged
NandhaKishorM merged 8 commits into
NandhaKishorM:mainfrom
keemsisi:feat/laya-java-runtime
Oct 4, 2026
Merged

NandhaKishorM merged 8 commits into
NandhaKishorM:mainfrom
keemsisi:feat/laya-java-runtime

Conversation

@keemsisi

@keemsisi keemsisi commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

laya-java: a JVM inference package, gated against the Python reference

The JVM has no laya runtime. laya-ts covers Node and the browser, laya-dotnet covers .NET, and
a Java or Kotlin service has to shell out to Python or stand up an HTTP hop for a decision that
costs a single forward pass. This adds laya-java: tokenizer, sequence builder, config, ONNX
inference, typed answers, predict / predictBatch, and usage accounting.

Nothing here is believed to match the Python package. All of it is measured to. Expectations are
generated from laya by a script in the repository, and a drift check fails when they move, so
the fixtures cannot quietly disagree with the code they came from. Precedent: sdk/typescript's
sync_presets.py --check, and the .NET lane's regen_golden.py.

 57 files changed, 36,116 insertions
 laya-java/laya-java/src/main : 22 files, 4,189 lines   (the runtime)
 laya-java/laya-java/src/test : 13 files, 1,833 lines   (241 tests)
 laya-java/scripts            :  3 files,   988 lines   (fixture generator, CI helpers)
 laya-java/fixtures           :  7 files,  612 KB       (generated, committed)
 laya-java/README.md          :    216 lines            (every example compiled on --release 17)
 .github/workflows            : java.yml, release-java.yml

Four commits, each self-contained: tokenizer + sequence builder → ONNX + decode + predict →
batching, usage, tests, CI and the review fixes → the README and the two defects writing it
found.


Current state of the gates

gate what it compares result
tokenizer fixture token ids on a 40-string hostile corpus × 2 checkpoints 80/80 byte-identical
tokenizer sweep (dev) 30,000 adversarial strings × 2 checkpoints vs HuggingFace 0 divergent
sequence fixture ids, markers, option stats, truncation report, 26 cases × 2 52/52 identical
decode fixture every answer field vs Agent._decode_answers, 20 × 2 40/40 identical
Python-JSON fixture CPython json.dumps spelling + round(v,4) 533 doubles, 11 structured, 1,691 rounding values, 0 divergent
real-model end to end 6 states / 10 questions + 3 batch configs vs ONNXAgent on the shipped 1.2 GB graph 9/9 identical
fixture drift gen_fixtures.py --check --strict clean
build javac -Xlint:all -Werror clean

241 tests, 0 failures, 0 aborted with the checkpoints present. 94 of them need no checkpoint
at all
and run anywhere; the other 147 abort by assumption with an actionable message, because a
skipped parity test is not a green one.


What the fixture-first approach caught, that writing from the format would have shipped

Every one of these was found by generating the expectation from Python before writing the Java,
and every one passes a reading of the tokenizer format that looks correct.

byte_fallback is all-or-nothing per CHARACTER, not per vocabulary. The multilingual
checkpoint ships 255 of the 256 <0xNN> byte tokens — <0x09>, the tab, is absent — and treating
one missing entry as "this model has no byte fallback" disabled the mechanism outright: 7,628 of
30,000 strings (25.4%)
went to <unk> where the reference emits byte tokens. The curated corpus
contained not one character outside the multilingual vocabulary, so it passed the broken version.
It now carries three such cases.

The byte-level alphabet check must cover 243 byte values, not 256. Thirteen — 0xC0, 0xC1
and 0xF5–0xFF — cannot occur in the UTF-8 encoding of any code point. The english checkpoint is
missing exactly those thirteen from its vocabulary and declares no unk_token, so a check over
all 256 rejects a shipped checkpoint. Verified by exhaustion over every valid code point
(U+0000–U+10FFFF less surrogates): 243 reachable, and the absent thirteen are precisely the
unreachable ones.

Metaspace must not prepend its marker when the text already starts with one. The multilingual
normalizer has already turned a leading space into U+2581, so prepending unconditionally invents
a token: " " encoded to two marker tokens instead of one, and " leading spaces" to five ids
instead of four.

Added tokens must match leftmost-LONGEST and carry their own ids. 253 of the english
checkpoint's added tokens and 1,335 of the multilingual one's are substrings of another, so
first-match cuts a 24-space run into twelve 2-space tokens. And 88 of english's 116 added tokens
are absent from model.vocab entirely
, with ids 50280–50367 above its maximum of 50279 —
[MASK] (50284) among them — so an id map built from model.vocab alone has no id for masked
prediction at all.

ignore_merges gates the whole-piece vocabulary shortcut, and both checkpoints declare it
false.
The shortcut is an attractive unconditional optimisation and an incorrect one. The twelve
multilingual vocabulary entries the merge loop cannot rebuild are all control tokens, which the
added-token stage resolves before BPE ever runs, so taking it has no upside either.

serialize_state and render_criterion feed the tokenizer, so Python's number spelling is part
of the contract.
Java's is not Python's: 1e+16 against 1.0E16, 1e-05 against 1.0E-5,
0.0001 against 1.0E-4. Python switches to scientific notation outside [1e-4, 1e16) where Java
switches outside [1e-3, 1e7), and JDK 17's Double.toString is not even guaranteed shortest — so
the digits are recovered by round-trip search rather than read off it.

Two of the bugs were in my own fixture generator, and both made the fixture agree with a port
that was wrong:

  • It resolved the tokenizer directory by searching for the shortest path containing a
    tokenizer.json, where laya resolves <model_dir>/tokenizer (onnx_agent.py:156,
    agent.py:235). The multilingual checkpoint ships a byte-identical copy at its root (same
    SHA-256) but only the subdirectory carries tokenizer_config.json, so the fixture recorded
    cls/sep/mask/pad as all None and I concluded the checkpoint shipped no special tokens. It ships
    four, and it reuses <bos> as CLS and <eos> as SEP (ids 2 and 1), which nothing about the
    tokenizer itself reveals.
  • render() passed sort_keys=True. Key order is not formatting here, it is data: a choice
    question's options render in the criteria map's insertion order, so the fixture asked a
    different question than the one measured, and a state dict came back alphabetised. Three sequence
    cases turned red on exactly this once the port was correct.

What an adversarial differential review caught

A review ran the Python reference and the Java port side by side over randomised inputs
(4,000 sequence cases, 6,000 tokenizer strings, 1,400 agent suites, the real graph on both sides).
Nine findings, all fixed, each with a test that fails without the fix.

1. CRITICAL — graph outputs were read by position from an unordered set, so they swapped in ~half of all JVMs

LayaSession asked for its outputs with Set.of("logits", "act_logits") and read them back by
index. OrtSession.Result's positional index follows the order of the requested set, and
Set.of with two or more elements randomises its iteration order per JVM process (its salt is
seeded from System.nanoTime()). Measured: 4 of 10 and 6 of 10 fresh JVMs gave each order.

It was silent on exactly the questions noul always produces: with two options both outputs are
two columns wide, so no shape check can notice and the answer simply inverts. On the real
multilingual graph, same code and same input:

Set.of order duplicate.noul confidence
[logits, act_logits] 0.9972 0.9972
[act_logits, logits] 0.0000 1.0000

Python's answer is 0.9972. So one order matched the reference exactly and the other reported
"the customer was charged more than once" as false at confidence 1.0000. With three options it
threw option count 3 does not fit a logit row of 2 instead. Python is immune because
onnx_agent.py:769 passes an ordered list.

Outputs are resolved by name now. Verified across six fresh JVMs: 9/9 each, where the same test
gave 0/9 in a pre-fix JVM.

This also corrects a misdiagnosis of mine earlier in development: I attributed the resulting flake
to fixtures being rewritten under a running test. They were not — the checksums were identical
before and after. It was this bug.

2. HIGH — a question whose markers did not all fit was answered partially instead of refused

Markers sit at absolute positions and the sequence is then cut to max_len, so with enough options
the later ones fall off. Java answered anyway, returning a distribution over the survivors with the
rest absent from the answer and unchoosable, while usage.truncated blamed the state. Python
raises (agent.py:1169). The guard is ported, naming both knobs because lowering head_max_len
can fix it while raising it makes it worse. This also closed two crash paths (p[1] on a one-marker
noul; a zero-marker question).

3. MEDIUM-HIGH — the questions map was re-read after inference, so a mutation could relabel an answer

Sequences are built before the graph runs; answers are labelled after. Reading the caller's map
twice let a mutation in between attach one question's labels to another question's logits — no
exception, nothing in usage. Snapshotted once at entry now, as Python does
(onnx_agent.py:735).

4. MEDIUM — two question shapes Python refuses were accepted

Blank or whitespace-only instructions (which asks the model nothing and gets a confident-looking
distribution anyway), and a null score level (which rendered the literal text level 0: null
into the input and returned legend={0: null}).

5. MEDIUM — a cyclic state raised StackOverflowError

An Error, so a server's catch (Exception) does not contain it, from caller-supplied data.
PythonJson now tracks the containers it is writing and refuses with CPython's own wording,
Circular reference detected. A container that merely appears twice is still written twice, which
is also what CPython does.

6. MEDIUM-LOW — added-token lstrip used Java's whitespace predicate, not Unicode's

Character.isWhitespace is not the Unicode White_Space property the reference strips: it excludes
U+0085, U+00A0, U+2007, U+202F and includes U+001C–U+001F. Eight characters disagree, so
"a" + U+00A0 + "[MASK]" gave three ids where HuggingFace gives two. This was the only tokenizer
divergence in 6,000 adversarial strings
, and not reachable through predict on the shipped
checkpoints — but it is reachable through the public Tokenizer.encode, and becomes reachable the
moment a checkpoint sets lstrip on an ordinary token. The property is now spelled out against the
standard, and single_word follows Rust's is_alphanumeric rather than Java's narrower
isLetterOrDigit.

7. LOW — close() was not idempotent, use-after-close gave a backend error, and a failing close leaked

ONNX Runtime throws on a second close, so try-with-resources plus an explicit close() threw. In
the split-graph form a failure closing the encoder left the head open. All three fixed; the first
failure is reported with the rest suppressed.

8. LOW — a state Map with non-String keys was refused, and a null state threw a bare NPE

CPython coerces {1: "x"} to {"1": "x"} and serialises None to the text null. Both
reproduced; a key type CPython itself refuses is still refused.

9. INFORMATIONAL — the drift gate could not pass without the 1.2 GB graph

Already fixed when the review landed, and it exposed a worse problem of mine: a family that cannot
regenerate records a skip, and writing that over a good committed fixture replaced a 107 KB file
with a 9 KB one
in my own testing. The generator now leaves such a file alone, --check reports
it unverified rather than stale, and --strict makes it an error for a lane that is supposed to
have the checkpoints. My first version of that guard required every section to be a skip, which
tokenizer_ids.json defeats because it always carries a corpus section — any section is enough.

What the review attacked and found nothing on

Worth a reviewer's attention, because it bounds where the risk is: sequence budget arithmetic
(4,000 randomised cases × 2 checkpoints, head_max_len 1–400 including <16, 1–60 options,
permuted orders — 0 diffs, with 2,846 hitting the re-cap path); the tokenizer beyond finding 6
(6,000 strings — byte_fallback, ignore_merges, astral/surrogate splitting, the 88 out-of-vocab
added tokens, leftmost-longest matching all correct); PythonJson float spelling (61 hard doubles
byte-identical); round4; decode arithmetic (only 45 of 309,482 numbers differ, every one by
exactly 1e-4 and all explained by the documented float32-vs-double softmax); batching, ordering and
usage across every batch size × sort order; thread safety (1,600 concurrent calls, byte-identical);
locale and charset (-Duser.language=tr -Dfile.encoding=US-ASCII — identical); and tensor lifetime
on the error path.


Precision and rounding: decided by measurement, not inheritance

The softmax runs in double, matching laya-ts and laya-dotnet. Python computes it in
float32 (the ONNX logits are float32 and NumPy keeps the dtype through the division under NEP
50). Computing in double changes the fourth decimal on 88 of 200,000 random logit rows — 1 in
2,300, which is agreement on prose and divergence on a customer's data. But matching float32
exactly is not available either: NumPy's own float32 exp differs from rounding a double exp on
3,239 of 500,006 values, and its pairwise summation differs from a sequential sum on 27% of
rows, surviving the rounding on 7 of 20,000. Neither is a contract NumPy publishes. So this
port holds the line the other two SDKs hold, and says so in the code rather than leaving it to be
found. All 40 decode cases and all 9 end-to-end cases agree under it.

Rounding went the other way, and the design note I wrote for this port had it wrong. It said to
match laya-ts's Math.round(v * 1e4) / 1e4 because the two never diverged across 1.76 million
softmax values. That measurement is real and reproduces here — over 1,316,901 softmax
probabilities, half-up and half-even both match CPython exactly. But Python's round is
half-to-even on the exact binary value, and over 200,000 exactly-representable .00005 steps
the laya-ts spelling misses 44,329 of them (22%). BigDecimal at scale 4 with HALF_EVEN
misses none, in every group measured. Exactness costs nothing here, so that is what the runtime
uses; the half-up spelling is kept only so the comparison is a test in this repository rather
than a claim in a comment.

Two contract details that the obvious reading gets wrong. act_probability is a softmax over
the head's second output, not the raw logit — reporting the logit publishes an unbounded real number
as a probability, and for a trained head it looks plausible. And the logit row is wider than the
option count, padded to the widest question in the batch with large finite numbers, so a softmax
over the whole row hands nearly all the mass to padding; the decode fixture feeds a deliberately
over-wide block so a port that forgets cannot pass.


Performance: no regression against the other SDKs

Best of 9 with 3 warm-ups discarded, same graph and same inputs on both sides, one machine. All
three runtimes produce 1,929 tokens on the corpus, which is the cross-check that they are doing
the same work.

Python (reference) laya-ts laya-java
tokenize a mixed corpus 1.06 ms · 1.83M tok/s 1.43 ms · 1.35M tok/s 1.59 ms · 1.22M tok/s
predict, 3 questions 124 ms — 95 ms
predict, long state (1,024 tokens) 3,644 ms — 3,208 ms
predictBatch, 8 states × 3 questions 887 ms — 700 ms

End to end laya-java is 12–23% faster than the Python reference; the tokenizer is within 11%
of laya-ts
— the right comparison, both being pure language code where the forward pass is ONNX
Runtime either way. Python's tokenizer is Rust and stays ahead, as expected.

The first measurement was bad, and both causes were mine.

  • Tokenizing that corpus took 24.5 ms against laya-ts's 1.4 ms — a 17× regression against
    another SDK of the same model.
    The BPE was the naive rescan, O(k²) in the length of a piece,
    which is the exact shape laya-ts had already fixed. It now routes long pieces through a min-heap
    over a doubly-linked list of surviving slots, O(k log k), keeping the rescan for short pieces where
    the heap's setup costs more than it saves. Getting it equivalent took one correction the fixture
    caught: a merge must bump the left neighbour's version as well as its own, or that neighbour's
    heap entry still carries a rank for a pair that no longer exists. Without it, one corpus entry in
    forty came out different. The 30,000-string sweep also dropped from 19.7 s to 13.5 s.
  • The session pinned setIntraOpNumThreads(1) and never set an optimisation level, so the forward
    pass ran on one core against an unoptimised graph. Python sets ORT_ENABLE_ALL and leaves the
    thread count to ONNX Runtime; this now does both. threads is still a parameter, for a
    request-per-thread server where the parallelism is already in the requests.

laya-dotnet could not be measured: there is no .NET toolchain on this machine. No claim is
made about it.


Design decisions

Decision Alternative Why not the alternative
In the monorepo, same com.convaiinnovations group a separate repository gen_fixtures.py runs the real Python in the same checkout. That is what makes the parity gate possible without a submodule or a published artifact to measure against.
One runtime dependency: onnxruntime bundle a JSON library A JSON dependency in the core forces its version on every consumer; two known schemas do not justify that. The internal reader is strict — it rejects what it does not understand rather than guessing, because a tokenizer parsed loosely produces ids no answer-level test can localise.
infer.InferenceSession seam call ONNX Runtime directly Batching, row offsets and usage accounting are the parts most likely to be wrong and least related to inference. The seam lets them be tested in milliseconds over a synthetic checkpoint, where the real graph is 1.2 GB and would keep them out of any CI that does not download one. It also stops OrtException appearing in every public signature.
Sealed Answer with three records one map, or one class with nullable fields A Choice is not a Score. A caller should not test a string to learn which fields are present, and the compiler should reject a missing case.
Both graph forms (fused laya.onnx, split encoder+head) fused only Both exist in the wild; laya-dotnet carries the same pair of paths. Which one a directory holds is discoverable, so there is no reason to make the caller say. (My design note described only the split form — the shipped exporter also produces the fused one, which is what ONNXAgent loads by default.)
LinkedHashMap everywhere order is semantic Map.copyOf Map.copyOf returns a map whose iteration order is unspecified, and it was silently discarding the option order in probabilities and legend. The fixtures now assert key order as an ordered list, not a set.
Fixtures generated and committed hand-written expectations Hand-written expectations in a port are how a tokenizer silently diverges. The drift check is what keeps the committed copy honest.
CI advisory (continue-on-error) blocking The Python package is the product and the port follows it; a Python-side improvement must not be blocked on a port catching up. Same reasoning, and same shape, as dotnet.yml.

Tests and CI

94 model-free tests — Python-compatible JSON spelling, four-decimal rounding, option rendering,
the collator, the config and its clamps, the Unicode whitespace property, and answer decoding
against expectations recorded from laya's own _decode_answers
— need no checkpoint and run
anywhere.

147 parity tests — tokenizer ids (86), sequence budgets (52), and the real-model end to end
(9) — abort by assumption without LAYA_CHECKPOINTS / LAYA_ONNX_GRAPH, with a message naming
what to set.

Batching and usage are tested without a model at all, over a synthetic byte-level checkpoint
written to a temp directory and a stub session. The stub keys its output on each row's content
rather than its index in the batch — keyed on the index, "the answers are the same at every batch
size" was trivially false, and the test caught that too.

java.yml follows the .NET lane: two jobs, advisory, fixtures re-recorded from the Python code at
each commit so a Python-side change shows up as drift, the checkpoint export cached separately from
the regeneration, and check_test_results.py failing a run whose parity tests only aborted —
because BUILD SUCCESSFUL with zero parity coverage is how a wrong answer sits behind a green
suite. Both CI helper scripts were self-tested against synthetic inputs (allow-aborted, strict,
below-floor, empty directory).


Publishing: configured, not done

The build produces signed, sources- and javadoc-bearing artifacts with the POM metadata Maven
Central requires — verified by generating the POM offline, with name, description, url,
Apache-2.0 (matching LICENSE, pyproject.toml, laya-ts and laya-dotnet), developers and
scm. release-java.yml mirrors the npm lane down to the manual environment gate, takes its
version from the tag so there is no second copy to drift, and refuses to push a -SNAPSHOT.

It cannot have been exercised, and only the project owner can finish it:

  1. Verify the com.convaiinnovations namespace at central.sonatype.com — Central will not accept
    a groupId whose domain or GitHub org has not been proven.
  2. Add MAVEN_CENTRAL_USERNAME / MAVEN_CENTRAL_PASSWORD (a Central portal token).
  3. Add LAYA_SIGNING_KEY / LAYA_SIGNING_PASSWORD (a PGP key, public half on a keyserver).

Signing activates only when a key is present, so ./gradlew build and publishToMavenLocal
work for anyone. publishToMavenLocal is tested; the Central step is not, and is marked as such in
both files. For reference, laya-dotnet is at the same stage: PackageId set, no publish workflow.


What I could not verify, stated plainly

  • ./gradlew build has not run on this machine. Gradle cannot resolve dependencies here: curl
    reaches Maven Central and the JVM times out on both IPv4 and IPv6, with no proxy configured. The
    build is validated as far as that allows — it configures, every publish task exists, and the POM
    generates offline — and the compiler was driven directly with the same -Xlint:all -Werror the
    build sets, which is how all 241 tests were run. CI will exercise the rest, and that is the first
    thing to watch on this PR.
  • The Maven Central publish step (above).
  • laya-dotnet performance — no .NET toolchain available.
  • Two committed fixtures currently gate nothing. lang_tables.json and presets.json are
    generated and committed, but no Java code consumes them yet — language detection and presets are
    Phase 5 and 7. They are harmless and they are staged for the follow-up, but they are not evidence
    of anything today, and I would rather say so than have them pad the fixture count. Happy to drop
    them from this PR if the maintainers would prefer nothing committed that is unused.
  • Wider docs are still thin. laya-java/README.md is in (quickstart, usage reporting,
    batching, build, parity, installing), and every code block in it is compiled on --release 17
    by a check I ran rather than eyeballed. There is no examples/ directory and no web demo
    equivalent, which laya-ts has.
  • Not implemented yet, by design, to keep this reviewable: hooks, the language router,
    predict_long, shortlist and structured output, the laya-java-client HTTP module (the Gradle
    module exists and is empty), and Android (explicitly out of scope for v1 — laya-ts's
    WebGPU/WASM path has no JVM analogue, and Android means a second ONNX Runtime distribution and a
    different threading model).

Two things writing the README caught

Documenting an API is a use of it, and this use found two defects that 241 tests had not.

Question.score(String, List<Object>) could not be called naturally. Java generics are
invariant, so that parameter rejects List.of("none", "minor") — the very first example in the
README did not compile. The tests had all declared Map<String, Object> and List<Object>
explicitly and so never hit it. The factories take Map<String, ?> and List<?> now.

The README claimed an exhaustive switch over the sealed Answer that does not compile on the
JDK this module targets.
Pattern matching for switch is preview in JDK 17 and final only in
21. The README now gives the portable instanceof form and says plainly that the exhaustive switch
is a language feature of 21, not of this library. The sealed interface still earns its place — the
three shapes are a closed set a reader can rely on.

Where to look first

  1. onnx/LayaSession.java — the output-by-name resolution, and the comment explaining why
    position was wrong. This is the one bug in the set that silently inverted answers.
  2. sequence/SequenceBuilder.java — the budget arithmetic, reproduced step for step from
    common.py:196, including the two rules that decide what survives: options yield nothing and the
    instruction yields everything, and the state silently takes only the room left after the head.
  3. tokenizer/Bpe.java — two routes to one rule (rescan and heap), and why.
  4. decode/Decoder.java — the precision note, which is the honest statement of where this port
    does not match Python bit for bit, and why no port can.
  5. scripts/gen_fixtures.py — the whole correctness argument lives here. If this is wrong, every
    green check above is worthless.
  6. README.md — if the quickstart does not read like something you would hand a user, say so;
    that is the part I have the least confidence in judging for this project.

Happy to split this into smaller PRs if the maintainers would rather review it in pieces — the
four commits are already self-contained along the obvious lines.

…gainst Python

The JVM has no laya runtime. `laya-ts` covers Node and the browser and `laya-dotnet` covers .NET,
so a Java service has to shell out to Python or stand up an HTTP hop for a decision that takes a
single forward pass. This adds the first two layers of a dependency-light JVM runtime -- tokenizer
and sequence builder -- behind generated parity fixtures, following the precedent `sdk/typescript`
set with `scripts/sync_presets.py --check`.

Nothing here is believed to match Python; all of it is measured to. `scripts/gen_fixtures.py`
generates the expectations FROM the library and `--check` fails when they drift, so the fixture
cannot quietly disagree with the code it came from. Current state: 80 tokenizer expectations and 52
sequence expectations byte-identical across both shipped checkpoints, plus a 30,000-string
adversarial sweep at zero divergence.

What the sweep bought, since the curated corpus alone did not catch any of it. Five bugs that a
port written from the obvious reading of the format would ship:

Metaspace prepends its marker only when the text does not already start with one. The multilingual
normaliser has already turned a leading space into U+2581, so prepending unconditionally invents a
token: " " encoded to two marker tokens instead of one, and "   leading spaces" to five ids instead
of four.

`byte_fallback` is all-or-nothing PER CHARACTER, not per vocabulary. The multilingual checkpoint
ships 255 of the 256 byte tokens -- <0x09>, the tab, is absent -- and treating one missing entry as
"this model has no byte fallback" disabled the mechanism outright: 7,628 of 30,000 strings went to
<unk> where the reference emits byte tokens, a 25.4% divergence. The curated corpus contained not
one character outside the multilingual vocabulary, so it passed the broken version; it now carries
three.

The byte-level alphabet check must cover the 243 byte values text can reach, not all 256. Thirteen
-- 0xC0, 0xC1 and 0xF5..0xFF -- cannot occur in the UTF-8 encoding of any code point, the english
checkpoint is missing exactly those thirteen, and it declares no unk_token, so a check over 256
rejects a shipped checkpoint. Verified by exhaustion over every valid code point: 243 reachable.

Added tokens must match leftmost-LONGEST and carry their own ids. 253 of the english checkpoint's
added tokens and 1,335 of the multilingual one's are substrings of another, so first-match cuts a
24-space run into twelve 2-space tokens. And 88 of english's 116 added tokens are absent from
`model.vocab` altogether, with ids 50280-50367 above its maximum of 50279 -- [MASK] (50284) among
them -- so an id map built from `model.vocab` alone has no id for masked prediction at all.

`ignore_merges` gates the whole-piece vocabulary shortcut, and both checkpoints declare it false.
The shortcut is an attractive unconditional optimisation and an incorrect one; the twelve
multilingual vocabulary entries the merge loop cannot rebuild are all control tokens, which the
added-token stage resolves before BPE, so taking it has no upside either.

Two fixture bugs of my own, both of which made the fixture agree with a port that was wrong:

The generator resolved the tokenizer directory by searching for the shortest path containing a
`tokenizer.json`, where laya resolves `<model_dir>/tokenizer` (onnx_agent.py:156, agent.py:235).
The multilingual checkpoint ships a byte-identical copy at its root -- same sha256 -- but only the
subdirectory carries `tokenizer_config.json`, so the fixture recorded cls/sep/mask/pad as all None
and I concluded the checkpoint shipped no special tokens. It ships four, and it reuses <bos> as CLS
and <eos> as SEP, which nothing in the tokenizer itself reveals.

`render()` passed `sort_keys=True`. Key order is not formatting here, it is data: a choice
question's options are rendered in the criteria map's INSERTION order, so the fixture asked a
different question than the one measured, and a state dict re-serialised in sorted order tokenizes
differently. Three sequence cases turned red on exactly this once the port was correct.

Also in: a JSON writer matching CPython's `json.dumps(ensure_ascii=False)`, because
`serialize_state` and `render_criterion` feed their output straight to the tokenizer, so every
character becomes token ids. Java's own spelling is not Python's -- 1e+16 against 1.0E16, 1e-05
against 1.0E-5, 0.0001 against 1.0E-4 -- Python switches to scientific notation outside
[1e-4, 1e16) where Java switches outside [1e-3, 1e7), and JDK 17's Double.toString is not even
guaranteed shortest, so the digits are recovered by round-trip search rather than read off it.
20,033 doubles including random bit patterns, denormals, both infinities and NaN, and 11 structured
values with control characters, U+2028/9 and arbitrary-precision integers: zero divergences.

The sequence builder reproduces `common.py:196` step for step, including the two rules that decide
what survives when a question does not fit -- options yield nothing and the instruction yields
everything, and the state silently takes only the room left after the head. It reports both, since
a caller cannot reconstruct a token budget that moves per checkpoint and per question. The option
collision counter is reported too: two options whose rendered text is identical for 48 tokens
collapse to the same span, the marker count still matches the option count, and nothing else
downstream can see it.

Not yet here: the ONNX session, answer derivation, batching, hooks, the router and predict_long.
No Android in v1.
…java

Completes the inference path: a collator, an ONNX Runtime session in both exported graph forms,
the typed answer decoder, and an `Agent` that answers several questions about one state in a single
forward pass. Gated by a new decode fixture family and by a real-model end-to-end run.

The decode fixture is driven through laya's OWN `Agent._decode_answers`, not a transcription of it,
over 20 synthesised logit rows per checkpoint chosen to hit the branches rather than to read well:
every temperature bucket, a tie at the top, a row whose softmax is exactly uniform, magnitudes of
+/-700 that overflow a naive exp, a logit block wider than the option count, permuted option
orders, and the per-language override. 40 answers byte-identical, including the key ORDER of
`probabilities` and `legend`.

Real-model end to end: the same six states and nine questions through Python's `ONNXAgent` on the
shipped 1.2 GB multilingual graph and through the Java `Agent`, comparing every field of every
answer. 6 of 6 identical. That is the only check covering tokenizer, sequence budgets, collation,
graph I/O and decode at once, and it is what the fixtures cannot do on their own.

PRECISION, measured rather than assumed, because this is where a port agrees on 99.9% of inputs and
ships the remaining 0.1%.

Python computes the softmax in FLOAT32 -- the ONNX logits are float32 and NumPy keeps the dtype
through the division under NEP 50. Computing it in double instead changes the fourth decimal on 88
of 200,000 random logit rows, which is 1 in 2,300: agreement on prose and divergence on a
customer's data. Matching float32 exactly is not available either. NumPy's own float32 `exp`
differs from rounding a double `exp` on 3,239 of 500,006 values, and its pairwise summation differs
from a sequential sum on 27% of rows, surviving the rounding on 7 of 20,000. Neither is a contract
NumPy publishes.

So this port does what `laya-ts` and `laya-dotnet` already do -- float32 logits, double softmax --
which is the standard the project has set, and the divergence is stated in the code rather than
discovered later. All 40 fixture cases and all 6 end-to-end cases agree under it.

Rounding went the other way, and the design note I wrote for this port had it wrong. It said to
match `laya-ts`'s `Math.round(v * 1e4) / 1e4` on the grounds that the two never diverged across
1.76 million softmax values. That measurement is real and reproduces here: over 1,316,901 softmax
probabilities, half-up and half-even both match CPython exactly, 0 mismatches each. But Python's
`round` is half-to-EVEN on the exact binary value, and over 200,000 exactly-representable .00005
steps the `laya-ts` spelling misses 44,329 of them -- 22% -- while `BigDecimal` at scale 4 with
HALF_EVEN misses none, in every group measured. Matching the reference exactly costs nothing here
and removes a "happens not to diverge" caveat instead of inheriting one, so `Rounding` uses
HALF_EVEN and keeps the half-up spelling only so the comparison is a test in this repository rather
than a claim in a comment.

Two contract details that would have been wrong from the obvious reading:

`act_probability` is a SOFTMAX over the head's second output, not the raw logit. Reporting the
logit publishes an unbounded real number as a probability, and for a well-trained head it looks
plausible, which is the worst kind of wrong. `Decoder.actionProbabilities` is separate from
`decode` for exactly that reason, since the decoder receives the row already normalised.

The logit row is WIDER than the option count. The head's output is padded to the widest question in
the batch and the padding columns are large finite numbers, not -inf, so a softmax over the whole
row hands nearly all the mass to padding. The row is sliced to the option count, and the fixture
feeds a deliberately over-wide block so a port that forgets cannot pass. The count itself is the
FILTERED marker count: a head that overran `max_len` loses markers, and the answer is derived over
the options that survived.

Also corrected here: the design note described only the split `encoder.onnx` + `head.onnx` export.
The shipped exporter also produces a FUSED `laya.onnx` taking all five inputs, which is what
`ONNXAgent` loads by default and what the end-to-end run above uses. `LayaSession` opens either,
preferring the fused one, as `laya-dotnet` does.

And `Map.copyOf` was quietly wrong for answers: it returns a map whose iteration order is
unspecified, and a choice's probabilities are positional. Both `probabilities` and `legend` now
come back in option order, which the fixture asserts as an ordered key list rather than a set.

Gates, all green: 80 tokenizer expectations, a 30,000-string adversarial tokenizer sweep, 52
sequence expectations, 40 decode expectations, 20,033 Python-JSON doubles, and 6 real-model
end-to-end cases -- every one byte-identical, plus the fixture drift check.

Still to come: predict_batch, hooks and usage accounting, the language router, predict_long,
shortlist and structured output, the client module, and docs.
…iew fixes

Adds `predictBatch` with usage accounting, moves the ONNX dependency behind a seam, puts the tests
in the repository where CI can run them, and fixes nine defects an adversarial differential review
found -- one of which made every answer a coin flip on JVM start.

241 tests pass, none aborted; the fixture drift check passes with --strict.

THE CRITICAL ONE. `LayaSession` asked the graph for its outputs with `Set.of("logits",
"act_logits")` and read them back by POSITION. `OrtSession.Result`'s index follows the order of the
requested set, and `Set.of` with two or more elements randomises its iteration order per JVM
process -- measured at 4 and 6 out of 10 fresh JVMs. So `logits` and `act_logits` were swapped in
about half of all processes. It was silent on exactly the questions `noul` always produces: with two
options both outputs are two columns wide, no shape check can notice, and the answer simply
inverts. On the real multilingual graph "the customer was charged more than once" came back TRUE at
0.9972 in one process and FALSE at confidence 1.0000 in the next, from the same code and the same
input. With three options it threw instead. Python is immune because it passes an ordered list.
Outputs are resolved by NAME now. Verified across six fresh JVMs: 9/9 each, where the same test was
0/9 in a pre-fix JVM.

That also explains a flake I had already seen and misdiagnosed as a fixture being rewritten under a
running test. It was not; the fixtures were byte-identical before and after.

The rest, each with a test that fails without the fix:

A question whose option markers did not all fit `max_len` was answered with a distribution over
whichever options survived -- the rest absent from the answer and unchoosable -- while
`usage.truncated` blamed the state. Python refuses the request; so does this, naming both knobs
because LOWERING `head_max_len` can fix it while raising it makes it worse.

The questions map was read again after inference to label the answers, so a mutation between encode
and decode attached one question's labels to another question's logits, with no exception and
nothing in `usage`. It is snapshotted once at entry now, as Python does.

Blank instructions and a null score level were accepted where Python refuses them: the first asks
the model nothing and gets a confident-looking answer anyway, the second renders the literal text
"level 0: null" into the input.

A cyclic state raised `StackOverflowError` -- an `Error`, which a server's `catch (Exception)` does
not contain -- from caller-supplied data. `PythonJson` tracks the containers it is writing and
refuses with CPython's own wording. A container that merely appears twice is still written twice,
which is also what CPython does.

Added-token `lstrip` used `Character.isWhitespace`, which is not the Unicode `White_Space`
property the reference tokenizer strips: eight characters disagree, so `"a" + U+00A0 + "[MASK]"`
gave three ids where HuggingFace gives two. The property is now spelled out against the standard,
and the `single_word` predicate follows Rust's `is_alphanumeric` rather than Java's narrower
`isLetterOrDigit`.

`close()` was not idempotent (ONNX Runtime throws on a second close, so try-with-resources plus an
explicit close threw), using a closed agent gave a backend error rather than an API one, and in the
split-graph form a failure closing the encoder left the head open. All three fixed; the first
failure is reported with the rest suppressed.

A state `Map` with non-String keys was refused where CPython coerces them, and a null state threw a
bare NPE where Python serialises `None` to the text "null".

PERFORMANCE, because a port that is right and three times slower is not a port anyone will use.
The first measurement was bad and both causes were mine.

Tokenizing a corpus with a 2,000-character run and a 1,000-character CJK run took 24.5 ms against
`laya-ts`'s 1.4 ms -- a 17x regression against another SDK of the same model. The BPE was the naive
rescan, O(k squared) in the length of a piece, which is the exact shape `laya-ts` had already fixed.
It now routes long pieces through a min-heap over a linked list of surviving slots, O(k log k),
keeping the rescan for short pieces where the heap's setup costs more than it saves. Getting it
equivalent took one correction the fixture caught: a merge must bump the LEFT neighbour's version as
well as its own, or that neighbour's heap entry still carries a rank for a pair that no longer
exists. Without it one corpus entry in forty came out different.

And the session pinned `setIntraOpNumThreads(1)` and never set an optimisation level, so the forward
pass ran on one core against an unoptimised graph. Python sets `ORT_ENABLE_ALL` and leaves the
thread count to ONNX Runtime; this now does both.

Measured after, best of 9 with 3 warm-ups discarded, same graph and inputs on both sides:

    tokenize a mixed corpus     Python 1.06 ms   laya-ts 1.43 ms   laya-java 1.59 ms
    predict, 3 questions        Python  124 ms                     laya-java   95 ms
    predict, long state         Python 3644 ms                     laya-java 3208 ms
    predict_batch, 8 x 3        Python  887 ms                     laya-java  700 ms

So end to end it is faster than the reference, and the tokenizer is within 11% of `laya-ts` -- the
right comparison, both being pure language code where the forward pass is ONNX Runtime either way.
Python's tokenizer is Rust and stays ahead. laya-dotnet could not be measured: no .NET toolchain on
this machine.

TESTS now live in the repository instead of beside it. 241 of them: the model-free ones
(Python-compatible JSON spelling, four-decimal rounding, option rendering, the collator, the config
and its clamps, and answer decoding against expectations recorded from laya's own
`_decode_answers`) need no checkpoint and run anywhere. The parity ones (tokenizer ids, sequence
budgets, and the real-model end to end) abort by assumption without `LAYA_CHECKPOINTS` /
`LAYA_ONNX_GRAPH` and say so, because a skipped parity test is not a green one.

Batching and usage are tested without a model at all, over a synthetic byte-level checkpoint and a
stub session, which is what the `infer.InferenceSession` seam is for: those are the parts most
likely to be wrong and least related to inference, and requiring a 1.2 GB graph would keep them out
of any CI that does not download one. The stub keys its output on each row's CONTENT rather than
its index in the batch -- keyed on the index, "the answers are the same at every batch size" was
trivially false, and the test caught that too.

The generator grew two safeguards of its own. A family that cannot regenerate records a skip, and
writing that over a good committed fixture replaced a 107 KB file with a 9 KB one in my own
testing; it now leaves the file alone and `--check` reports it unverified rather than stale, with
`--strict` for a lane that is supposed to have the checkpoints. The first version of that guard
required EVERY section to be a skip, which `tokenizer_ids.json` defeats because it always carries a
corpus section -- any section is enough.

CI follows the .NET lane: advisory rather than blocking, because the Python package is the product
and a port must not block it; fixtures re-recorded from the Python code at each commit so a
Python-side change shows up here as drift; the checkpoint export cached separately from the
regeneration; and a results check that fails a run whose parity tests only aborted.

PUBLISHING is configured but not done. The build produces signed, sources- and javadoc-bearing
artifacts with the POM metadata Maven Central requires, and `release-java.yml` mirrors the npm
lane's shape down to the manual environment gate. It cannot have been exercised: Central needs the
`com.convaiinnovations` namespace verified by the project owner, a portal token and a PGP key, none
of which are in this repository. Signing activates only when a key is present, so `./gradlew build`
and `publishToMavenLocal` work for anyone. `publishToMavenLocal` is tested; the Central step is
not, and is marked as such in both files.

One environment note, not a defect: Gradle cannot resolve dependencies on the machine this was
written on -- `curl` reaches Maven Central and the JVM times out on both IPv4 and IPv6 -- so
`./gradlew build` has not run here. The build file is validated as far as that allows (it
configures, and the POM generates offline with every required field), and the compiler was driven
directly with the same `-Xlint:all -Werror` the build sets. CI will exercise the rest.

Still to come: hooks, the language router, predict_long, shortlist and structured output, the
client module, and docs.
@keemsisi
keemsisi marked this pull request as draft October 4, 2026 15:56
@keemsisi keemsisi changed the title Feat/laya java runtime eat(java): add laya-java JVM inference package Oct 4, 2026
@keemsisi keemsisi changed the title eat(java): add laya-java JVM inference package feat(java): add laya-java JVM inference package Oct 4, 2026
…t found

Documenting an API is a use of it, and this use found two defects that 241 tests had not.

`Question.score(String, List<Object>)` could not be called naturally. Java generics are invariant,
so that parameter rejects `List.of("none", "minor")` -- the very first example in the README did not
compile. Every test had declared `Map<String, Object>` and `List<Object>` explicitly and so never
hit it. The factories take `Map<String, ?>` and `List<?>` now, which is the whole fix.

And the README's first draft claimed an exhaustive `switch` over the sealed `Answer`. That does not
compile on the JDK this module targets: pattern matching for `switch` is a PREVIEW feature in 17 and
final only in 21. The README gives the portable `instanceof` form and says plainly that the
exhaustive switch belongs to the language version, not to this library. The sealed interface still
earns its place -- the three shapes are a closed set a reader can rely on, and `instanceof` needs no
`else` that throws.

Every code block in the README is compiled against the real classes on `--release 17` rather than
eyeballed, which is how both of the above surfaced.

The README covers what exists and says what does not: exporting a graph, asking questions, the two
confidences and why they must not share a threshold, the usage and truncation report, batching, the
lower-level pieces, threads, build and test (including the environment variables the parity tests
need), how the fixtures are generated and where this port does not match Python bit for bit, and
installing -- which is `publishToMavenLocal`, because nothing is published yet.

241 tests still pass; the fixture drift check is clean.
@keemsisi keemsisi changed the title feat(java): add laya-java JVM inference package feat(java): laya-java: a JVM inference package, gated against the Python reference Oct 4, 2026
…chine-dependent

The Java lane went red on its first run. Four causes, all mine.

A COMMITTED FIXTURE DEPENDED ON libm. `gen_fixtures.py --check` reported `python_json.json` stale on
the Linux runner while passing on macOS, because that family built its random doubles with
`10 ** rng.randint(-320, 300) * rng.random()` and its probability corpus with `math.exp`. Both are C
library calls, and neither is bit-identical across platforms -- `pow` into the subnormal range and
`exp` are exactly the routines that differ by an ulp between implementations. A fixture that is a
property of the machine that generated it is not a fixture.

Every value in that family now comes from an exact route: a decimal literal (correctly rounded by
strtod), IEEE arithmetic, integer division, or a raw 64-bit pattern. `random()` is exact -- it
scales Mersenne Twister integers by a power of two -- and a bit pattern reaches the subnormals, the
infinities and the NaNs that arithmetic would not. The distribution group is built as
`weight / sum(weights)` over random integers, which is one correctly-rounded division and still
produces the normalised, unevenly-spread values a softmax produces, which is all the rounding rule
cares about. `math.pi` and `math.e` are written out as literals so the family carries no `math`
reference at all. The group is renamed from `softmax` to `distribution`, because calling it a
softmax when it is not would be the kind of comment that rots.

THE EXPORT SUBPROCESS COULD NOT IMPORT laya. `prepare_checkpoint.py` put the repository on its own
`sys.path` and then ran `scripts/export_onnx.py` as a subprocess, which failed with
ModuleNotFoundError: a script's `sys.path[0]` is the SCRIPT's directory, so the exporter got
`<repo>/scripts` and never the root, cwd notwithstanding. It passes PYTHONPATH now, as the .NET
lane's regeneration does.

This one could not be reproduced locally, and the reason is worth recording: the venv here carries
a stale editable install of `laya` pointing at a different checkout, so `import laya` succeeded and
silently resolved to the WRONG TREE. CI, with no install, is the honest environment. Setting
PYTHONPATH fixes the failure there and stops a local run measuring something other than the working
tree here.

A SKIPPED TEST STEP CRASHED THE RESULTS CHECK. `check_test_results.py` runs under
`if: !cancelled()` so that an aborted suite cannot pass unnoticed, which means it also runs when an
earlier step failed and the test step was therefore skipped. It then raised FileNotFoundError on
the missing results directory and buried the real failure under a stack trace from the wrong
script. It now says that nothing ran and that the failure is upstream.

AND THE JOB NAMES COLLIDED. `dotnet.yml` already has "build + model-free tests" and
"parity (english)", so the pull request's checks list showed each name twice with one red and one
green and no way to tell which lane was which. The Java jobs are prefixed `jvm` now; a check across
every workflow confirms no duplicate job name remains.

One ordering change while here: the model-free drift gate runs LAST, not first. It and the tests are
independent signals -- the tests compare against the committed fixture, the gate compares the
committed fixture against a fresh recording -- and running the gate first meant a stale fixture
skipped the build and the tests entirely, so the run said nothing at all about whether the code
worked.

241 tests still pass; `--check --strict` is clean.
The first fix got the export working and the lane went red one step later: `gen_fixtures.py
--strict` reported `tokenizer_ids.json`, `sequences.json` and `decode.json` as families it could
not regenerate.

The cause is a shape mistake I made copying the .NET lane. That lane matrixes over checkpoints
because its goldens live in per-checkpoint directories. These fixtures do not: each file carries
BOTH checkpoints' sections in one document, so a job holding only `english` can never regenerate
them and `--strict` fails by construction, in every job, forever. The matrix is gone; one `jvm
parity` job prepares both.

Only the checkpoint the end-to-end golden was recorded from needs a graph, and therefore weights,
so `prepare_checkpoint.py` takes `--no-graph` to fetch just the tokenizer and config. Running it
caught a second bug: it was copying `snapshot_download`'s return value, which is the CACHE ROOT,
and `allow_patterns` governs only what gets downloaded -- anything already cached is still sitting
there. The english checkpoint has no subfolder, so its directory IS the repository root and
contains `multilingual/` and `typed-decisions/` as children: a "tokenizer only" preparation
produced 2.2 GB. It downloads into a `local_dir` now and moves the wanted subtree into place, which
is 3.6 MB, and it asserts the layout rather than trusting it. Verified end to end: Java loads a
weights-free checkpoint (vocab 50,368, cls 50281, mask 50284, max_len 512) and the generator
records english from it.

The downloaded checkpoints and exported graphs now live under `laya-java/.work/` instead of the
repository root, so one `.gitignore` covers them and the root stays clean.

Also removed two imports the previous commit left dead (`re`, and the `math` that went with the
libm fix), which CI's ruff selection would have caught.

241 tests pass, `--check --strict` is clean, every script's CLI runs, and the YAML parses with no
duplicate job name across any workflow.
…rom CPython

Groundwork for language detection, and it has to come first because the JDK disagrees with CPython
about every character property `laya/lang.py` depends on. Measured over all 1,114,112 code points:

`str.isalpha()` is categories Lu/Ll/Lt/Lm/Lo, which is exactly what `Character.isLetter` means --
and the JDK still misses 4,863 of the code points Python calls letters, because JDK 17 ships
Unicode 13.0 and CPython 3.12 ships 15.0. U+0870 (Arabic Extended-B), U+1E290 (Toto) and U+11F00
(Kawi) come back UNASSIGNED. `_script_counts` skips anything that is not alphabetic, so text in a
recently-added script would be counted under no script at all, make the total zero, detect as
"unknown", and `analyse` treats unknown as English: the request would go to the checkpoint that has
no tokens for it. Silently.

Python's `[^\W\d_]`, which is what the word regex matches, is letters plus the Nl and No numeric
categories. Same version gap, 4,883 code points.

`unicodedata.combining(ch) != 0` is the canonical combining CLASS, and the JDK does not expose it
at all. The .NET port approximates it with the Mn and Mc categories; that approximation is wrong on
1,460 code points -- 1,410 it calls combining that Python does not, including U+034F COMBINING
GRAPHEME JOINER whose class is zero, and 50 it misses. `_non_latin_words` uses it to decide where a
word breaks, so this is not cosmetic: a wrong break turns one capitalised name into two lowercase
runs, and a lowercase run of two or more letters is what makes text "non-Latin words" rather than
annotation.

So the three tables are recorded from CPython into generated Java source: 659 ranges for isalpha,
709 for the word class, 192 for the combining class, about 29 KB with a binary search over flat
`int[]` pairs. That also makes detection independent of the JDK a consumer happens to run, which
matters more than it sounds -- a JDK upgrade must not quietly change which script a request is
detected as -- and `gen_unicode_tables.py --check` fails when a CPython upgrade moves them, so the
change arrives as a red lane rather than as a routing difference nobody attributed.

AND `str.lower()` IS NOT `String.toLowerCase`. Java applies the contextual rules, so
"ΑΣ" lowercases to "ας" -- a FINAL sigma -- where Python gives
"ασ". `latin_profile` lowercases the whole text before matching stopwords, so a Greek
word ending a sentence would match differently. Lowercasing one code point at a time removes the
context and therefore the rule; Unicode 15 has exactly ONE code point whose lowercase is longer
than one code point (U+0130, which `laya/lang.py` already special-cases for its own reasons), and
that one is handled explicitly.

Verified exhaustively rather than argued: all three predicates and `pythonLower` agree with CPython
15.0.0 on every code point, 0 extra and 0 missing, and the final-sigma case now matches.

No detection logic yet; this is the layer it stands on.
Finishes the groundwork for language detection. Three more JDK-versus-CPython gaps, each measured
over all 1,114,112 code points, and each one reachable from a routing decision:

Python's `\d` is the Nd category and the JDK misses 30 of them (U+11F50, the Kawi digits, Unicode
15). It matters because Python's `\w` is letters plus EVERY numeric category, and the three regexes
in `laya/lang.py` subtract from it: `[^\W\d_]` is the word class, `[^\W_]` is what the joined-token
pattern reads. Without the digit table those subtractions are wrong.

`str.isupper()` and `str.islower()` are missing 40 and 200 code points respectively -- U+10FC, a
Georgian modifier letter Python calls lowercase through Other_Lowercase, is the first. Detection
reads the case of the FIRST letter of a non-Latin run to tell a proper noun from a request
(`Дмитрий` is a name, `дмитрий` is prose), and it tests whether a segment has any lowercase letter
at all before deciding an all-caps token is an acronym. A letter whose case the JDK does not know
would change both.

So UnicodeTables now carries six tables -- isalpha, the word class, the combining class, digits,
uppercase and lowercase -- and the string-level helpers Python's methods imply: `isUpperString` (at
least one cased code point, every cased one uppercase, so titlecase does NOT count as upper) and
`hasLower`. Verified against CPython on every code point, 0 extra and 0 missing on all six, plus 19
string-level cases including U+01C5 titlecase, the sharp s, the dotted capital I and the Roman
numerals.

Also generated: LanguageTables, the word lists and script ranges `laya/lang.py` detects with -- 25
script ranges, 11 stopword lists totalling 671 words, and six smaller sets. Compiled in rather than
read from `fixtures/lang_tables.json`, because a runtime library must not need a test fixture on
disk; the fixture keeps its job, since a test will assert the compiled copy equals it. That is also
the answer to a review note on the first push: `lang_tables.json` was committed and gated nothing.
It gates this.

Two orderings in that file are load-bearing and preserved deliberately. `SCRIPT_RANGES` is scanned
in order and the first matching range wins, so a reordering changes which script a code point
counts under. And `STOP_WORDS` is a LinkedHashMap because the per-language scores are compared with
`max`, which returns the FIRST of equal values -- so its order decides ties between languages that
scored the same.

Both generators have `--check`, wired into the lane, so a CPython upgrade or a change to
`laya/lang.py` arrives as a red check rather than as a routing difference nobody attributed.

Still no detection logic; next.
@NandhaKishorM
NandhaKishorM merged commit 11b7f1e into NandhaKishorM:main Oct 4, 2026
30 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged for 0.3.27.

You arrived with the convention already applied, which is the reason this took one pass instead of three. The .NET lane became advisory yesterday in 0.3.26 and your header cites that reasoning back at me ("the Python package is the product and laya-java follows it, so a Python-side improvement must not be blocked on a port catching up"), with continue-on-error on both jobs. I did not have to ask for anything.

"Nothing here is believed to match the Python package. All of it is measured to" is the right sentence for a port, and the structure behind it is what I checked:

  • fixtures generated from laya by a committed script (scripts/gen_fixtures.py) rather than recorded once and trusted;
  • a drift check so they cannot quietly disagree with the code they came from;
  • check_test_results.py as the skip guard, so a missing artifact fails the run instead of passing as green. That is the failure mode that makes a parity lane decorative, and it is the third port in a row to get it right.

Citing sdk/typescript's sync_presets.py --check and the .NET regen_golden.py as precedent is also the right instinct: three ports converging on one verification pattern is worth more than three clever ones.

On the release workflow, which is the part I looked at hardest because #671 deliberately left publishing out. Taking it is fine here because it cannot fire by accident:

  • it triggers on laya-java-v*, which is disjoint from the v* the Python release uses, so tagging v0.3.27 does not touch it (I verified both patterns);
  • the one-time owner setup it needs (Central namespace verification, portal token, PGP key) does not exist, so it cannot run today;
  • build.gradle.kts activates signing only when a key is present, so ./gradlew build and publishToMavenLocal work for anyone without it.

Documenting the three setup steps in the workflow header, where whoever does them will look, is better than a README section nobody finds.

What I am not claiming: I have not reviewed the Java to the standard I hold laya/ to, there is no Maven artifact yet, and I have not run the lane locally (no JDK here). The parity lane going green on main is the evidence, same as for .NET.

Verified: the shared suite list reports all 79 suites passed, laya/ is untouched by this PR, and a decision diff against 0.3.26 on the real english and multilingual checkpoints shows zero decision flips.

Thank you.

@keemsisi

keemsisi commented Oct 4, 2026

Copy link
Copy Markdown
Contributor Author

Merged for 0.3.27.

You arrived with the convention already applied, which is the reason this took one pass instead of three. The .NET lane became advisory yesterday in 0.3.26 and your header cites that reasoning back at me ("the Python package is the product and laya-java follows it, so a Python-side improvement must not be blocked on a port catching up"), with continue-on-error on both jobs. I did not have to ask for anything.

"Nothing here is believed to match the Python package. All of it is measured to" is the right sentence for a port, and the structure behind it is what I checked:

  • fixtures generated from laya by a committed script (scripts/gen_fixtures.py) rather than recorded once and trusted;
  • a drift check so they cannot quietly disagree with the code they came from;
  • check_test_results.py as the skip guard, so a missing artifact fails the run instead of passing as green. That is the failure mode that makes a parity lane decorative, and it is the third port in a row to get it right.

Citing sdk/typescript's sync_presets.py --check and the .NET regen_golden.py as precedent is also the right instinct: three ports converging on one verification pattern is worth more than three clever ones.

On the release workflow, which is the part I looked at hardest because #671 deliberately left publishing out. Taking it is fine here because it cannot fire by accident:

  • it triggers on laya-java-v*, which is disjoint from the v* the Python release uses, so tagging v0.3.27 does not touch it (I verified both patterns);
  • the one-time owner setup it needs (Central namespace verification, portal token, PGP key) does not exist, so it cannot run today;
  • build.gradle.kts activates signing only when a key is present, so ./gradlew build and publishToMavenLocal work for anyone without it.

Documenting the three setup steps in the workflow header, where whoever does them will look, is better than a README section nobody finds.

What I am not claiming: I have not reviewed the Java to the standard I hold laya/ to, there is no Maven artifact yet, and I have not run the lane locally (no JDK here). The parity lane going green on main is the evidence, same as for .NET.

Verified: the shared suite list reports all 79 suites passed, laya/ is untouched by this PR, and a decision diff against 0.3.26 on the real english and multilingual checkpoints shows zero decision flips.

Thank you.

Thanks for merging #927. That one covers the runtime; the routing layer is in a follow-up PR

On the Python in laya-java/scripts/: those are test generators, not runtime. They run the laya package and record its output into laya-java/fixtures/*.json, and the Java tests assert against those files — so the port is tested and benchmarked against Python rather than against values I wrote by hand. That is what caught Character.toLowerCase silently failing on 40 code points CPython maps (JDK 17 ships Unicode 13, CPython 3.12 ships 15); a hand-written test would have agreed with whatever I wrote.

The SDK itself needs no Python: the fixtures are committed, nothing in src/ starts a process, and the full suite passes on a PATH with no python on it. Python is only for regenerating those files, and in CI to check the committed copies still match the reference, so drift fails a check instead of going unnoticed.

keemsisi added a commit to keemsisi/laya that referenced this pull request Oct 4, 2026
Two more adversarial passes, this time over the code that merged in NandhaKishorM#927 and
had only ever had mutation testing and parity fixtures. 25 findings; the
release-blocking ones first, because any of them could have done damage that
cannot be undone.

RELEASE AND CI

  * release-java.yml on a manual run from a BRANCH published to Maven Central
    with the branch name as the version. GITHUB_REF_NAME is the branch there,
    so the version became `main`; the snapshot guard only matched
    `*-SNAPSHOT` and let it through; and the build job's version check was
    guarded by a tag condition the publish job did not share, so the two jobs
    disagreed about what was being released and the artifact published was
    never the one the tests ran against. Central is append-only: `:main` could
    never have been replaced. It now refuses to publish from anything but a
    tag.
  * the release gate's test floor was 80 against a 484-test model-free suite,
    so 83% of the tests could vanish and the publish still went ahead. Now
    440, matching the build lane.
  * check_test_results.py compared the floor against JUnit's `tests`
    attribute, which COUNTS SKIPPED TESTS. A suite in which every single test
    aborted satisfied a floor of 440 and both --allow-aborted lanes -- the
    build gate and the Central release gate -- went green with nothing
    executed. That is the exact failure the script's docstring says it exists
    to catch. It counts executed tests now, and an all-skipped suite fails.
  * the empty `laya-java-client` module was in the Gradle build with a full
    MavenPublication, so a release would have pushed a zero-class jar to
    Central with a POM promising an HTTP client the README itself lists as not
    implemented. It joins the build when it has code.
  * publishing required two of the four secrets. Without the signing
    passphrase the build calls useInMemoryPgpKeys(key, null) and uploads
    UNSIGNED artifacts, which Central rejects only after the upload.

THE SPLIT GRAPH LAYOUT WAS DEAD. `encoder.onnx` + `head.onnx` is a
documented, advertised layout, and the head declares `qtype` as [B, 1] while
this passed a rank-1 tensor -- so ORT refused every graph the only exporter
for that layout produces, and the error surfaced as a per-batch backend
failure that reads like a bad model rather than a bad client. laya-ts passes
[n, 1] for the same reason.

OTHER RUNTIME DEFECTS

  * a temperature written as a JSON string was discarded and silently
    defaulted to 1.0, where the reference parses it. Measured: the published
    answer_confidence -- the number callers gate on -- was overstated by 19
    points with no warning, in a class whose whole stated purpose is that
    these are read from the checkpoint and never defaulted.
  * a temperature of the wrong TYPE (a number, a string, a mapping) loaded
    cleanly at 1.0 where the reference refuses the checkpoint. The wrong
    LENGTH was already refused; only the type slipped through.
  * a logits output of the wrong rank or dtype escaped as a raw
    ClassCastException naming neither the graph nor the output, through a
    method whose javadoc promises the backend never leaks -- so a caller's
    catch (InferenceException) did not catch it. An fp16 or fp64 re-export
    reaches it.
  * run() never consulted `closed`, so the guard isClosed()'s own javadoc
    promises ("a caller gets an API error, not a backend one") did not exist,
    and a use-after-close produced exactly the error class the API says it
    never emits.
  * the action softmax padded a one-column head up to two columns, turning a
    probability of 1.0 into a sigmoid -- a plausible-looking number for a
    value the model never produced, in the field callers gate escalation on --
    and published NaN for a zero-column row. It uses the graph's own width.
  * round4 lost the sign of a negative zero, so the exact-parity claim did not
    hold and a record's equals() would have reported two equal answers as
    different.

THE JSON LAYER, which parses every fixture and every tokenizer.json

  * no depth limit in either direction: deeply nested input threw
    StackOverflowError -- an Error, which a server's catch (Exception) does
    not contain -- where CPython raises a catchable RecursionError. The
    identity-set fix had closed only the cyclic case. Both now have one.
  * an integer literal was converted with a QUADRATIC BigDecimal call and no
    digit cap: a 1 MB literal cost 21 seconds of a request thread, from one
    field of a checkpoint downloaded off the hub. Capped at CPython's 4,300.
  * a backslash-u escape accepted fullwidth and Arabic-Indic digits as hex,
    because Character.digit is Unicode-aware, so two byte-different documents
    parsed to the same value. ASCII only now.

481 tests pass, every generator gate is clean, and one test that had been
asserting the temperature defect was corrected rather than the fix weakened.
keemsisi added a commit to keemsisi/laya that referenced this pull request Oct 5, 2026
Two more adversarial passes, this time over the code that merged in NandhaKishorM#927 and
had only ever had mutation testing and parity fixtures. 25 findings; the
release-blocking ones first, because any of them could have done damage that
cannot be undone.

RELEASE AND CI

  * release-java.yml on a manual run from a BRANCH published to Maven Central
    with the branch name as the version. GITHUB_REF_NAME is the branch there,
    so the version became `main`; the snapshot guard only matched
    `*-SNAPSHOT` and let it through; and the build job's version check was
    guarded by a tag condition the publish job did not share, so the two jobs
    disagreed about what was being released and the artifact published was
    never the one the tests ran against. Central is append-only: `:main` could
    never have been replaced. It now refuses to publish from anything but a
    tag.
  * the release gate's test floor was 80 against a 484-test model-free suite,
    so 83% of the tests could vanish and the publish still went ahead. Now
    440, matching the build lane.
  * check_test_results.py compared the floor against JUnit's `tests`
    attribute, which COUNTS SKIPPED TESTS. A suite in which every single test
    aborted satisfied a floor of 440 and both --allow-aborted lanes -- the
    build gate and the Central release gate -- went green with nothing
    executed. That is the exact failure the script's docstring says it exists
    to catch. It counts executed tests now, and an all-skipped suite fails.
  * the empty `laya-java-client` module was in the Gradle build with a full
    MavenPublication, so a release would have pushed a zero-class jar to
    Central with a POM promising an HTTP client the README itself lists as not
    implemented. It joins the build when it has code.
  * publishing required two of the four secrets. Without the signing
    passphrase the build calls useInMemoryPgpKeys(key, null) and uploads
    UNSIGNED artifacts, which Central rejects only after the upload.

THE SPLIT GRAPH LAYOUT WAS DEAD. `encoder.onnx` + `head.onnx` is a
documented, advertised layout, and the head declares `qtype` as [B, 1] while
this passed a rank-1 tensor -- so ORT refused every graph the only exporter
for that layout produces, and the error surfaced as a per-batch backend
failure that reads like a bad model rather than a bad client. laya-ts passes
[n, 1] for the same reason.

OTHER RUNTIME DEFECTS

  * a temperature written as a JSON string was discarded and silently
    defaulted to 1.0, where the reference parses it. Measured: the published
    answer_confidence -- the number callers gate on -- was overstated by 19
    points with no warning, in a class whose whole stated purpose is that
    these are read from the checkpoint and never defaulted.
  * a temperature of the wrong TYPE (a number, a string, a mapping) loaded
    cleanly at 1.0 where the reference refuses the checkpoint. The wrong
    LENGTH was already refused; only the type slipped through.
  * a logits output of the wrong rank or dtype escaped as a raw
    ClassCastException naming neither the graph nor the output, through a
    method whose javadoc promises the backend never leaks -- so a caller's
    catch (InferenceException) did not catch it. An fp16 or fp64 re-export
    reaches it.
  * run() never consulted `closed`, so the guard isClosed()'s own javadoc
    promises ("a caller gets an API error, not a backend one") did not exist,
    and a use-after-close produced exactly the error class the API says it
    never emits.
  * the action softmax padded a one-column head up to two columns, turning a
    probability of 1.0 into a sigmoid -- a plausible-looking number for a
    value the model never produced, in the field callers gate escalation on --
    and published NaN for a zero-column row. It uses the graph's own width.
  * round4 lost the sign of a negative zero, so the exact-parity claim did not
    hold and a record's equals() would have reported two equal answers as
    different.

THE JSON LAYER, which parses every fixture and every tokenizer.json

  * no depth limit in either direction: deeply nested input threw
    StackOverflowError -- an Error, which a server's catch (Exception) does
    not contain -- where CPython raises a catchable RecursionError. The
    identity-set fix had closed only the cyclic case. Both now have one.
  * an integer literal was converted with a QUADRATIC BigDecimal call and no
    digit cap: a 1 MB literal cost 21 seconds of a request thread, from one
    field of a checkpoint downloaded off the hub. Capped at CPython's 4,300.
  * a backslash-u escape accepted fullwidth and Arabic-Indic digits as hex,
    because Character.digit is Unicode-aware, so two byte-different documents
    parsed to the same value. ASCII only now.

481 tests pass, every generator gate is clean, and one test that had been
asserting the temperature defect was corrected rather than the fix weakened.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants