Repository navigation
feat(java): language routing, presets and the embedding shortlist - #938
Conversation
|
Holding this for one revision, on a defect rather than on scope. Everything else below is a note, this part is the blocker. The release gate is disabled by the change that was meant to tighten it
The shell therefore runs: Both arguments are gone. This is the only place it happens. The same change in I would not normally hold a 35,000 line PR on four lines, but this one publishes to Maven Central and Central is append-only, and the PR's own argument is that a floor which does not bind is not a gate. It is worth getting right before it lands. The rest of it reviewed well, so this is closeThe four defects you lead with are real and I checked three of them on
The list-state truncation bug,
The BLAS tolerance is handled the way I would want: measured rather than conceded, twelve decimals recorded, a test that fails if the recorded precision and the comparison stop agreeing, and the label order, which is the only thing a consumer observes, kept exact. Noting that recording the scores verbatim would have made the fixture disagree with itself between macOS and OpenBLAS is the part that explains why this is not laziness. 82 of 86 mutants dying, with each of the four survivors turned into a guarded invariant that fails when its enabling property changes, and the rounding mode pinned by a fixture because it is unobservable through Two more things, neither a blockerIt will conflict with The The Fix the folded block and I will take this. |
Routing decides whether a state goes to the English checkpoint or the
multilingual one, and the English checkpoint collapses to near-random on a
script it has no tokens for (0.100 on Hindi at 20 options, where random is
0.050). So this is the part of the port where a small difference is a wrong
answer rather than a rounding discrepancy.
What the reference leans on, and what Java gives instead:
* four regexes. Java's \w is not Python's -- it admits combining marks and
join controls -- and \d and \p{L} follow the JDK's Unicode version rather
than CPython's. Each pattern is a hand-written scan over the recorded
tables. The identifier scan reproduces the reference's lookbehind, which
is what keeps it linear: without it a 4,000-character token costs 205 ms
against 0.45 ms for prose, and a state is user input.
* str.lower(). Not String.toLowerCase, which applies the contextual
final-sigma rule; and not Character.toLowerCase either, which at Unicode
13.0 does not know the lowercase of 40 code points CPython 15.0 maps
(all of Vithkuqi among them) and returns them unchanged. A table, so the
mapping does not depend on the JDK a consumer runs.
* str.isspace(), which strip() and split() work on. Character.isWhitespace
rejects four of its members -- U+0085, U+00A0, U+2007, U+202F -- and
Character.isSpaceChar is wrong the other way. A no-break space pasted
from a word processor otherwise fuses two words into one token that no
stopword list holds.
* dict insertion order, which is data here: it orders the public script
profile and breaks every tie between two languages with the same score.
LinkedHashMap throughout, and the fixture now records the stopword order
explicitly -- it was being sorted away, which is the same mistake the
generator's missing sort_keys was fixed for once already.
Faster than the reference and than laya-ts, on the same corpus, best-of-7
after three warmups:
Python laya-ts laya-java
1,700 analyse calls 85 ms 34 ms 25 ms
20 calls on a 3.8 KB state 37 ms 14 ms 8 ms
Two changes earn most of that, and neither costs a divergence. Script lookup
is a binary search over one sorted table instead of a scan over 25 scripts in
order -- equivalent because no two ranges overlap, which the generator now
refuses to emit a table without, and which a test proves by sweeping every
code point. And each character predicate answers ASCII from a pair of 64-bit
masks computed from the same predicate as its range table, instead of ten-odd
comparisons. Counting letters by script now tallies into an int[] with no
boxing or hashing per character.
Gated by 137 tests. The corpus is 106 cases recorded from the reference, each
asserting all eight output fields and, for a string, all fourteen
intermediate steps, so a failure names the stage that broke. The character
tables are compared against CPython by a sha256 over all 1,114,112 code
points, nine digests in 500 bytes of fixture -- replacing a hand check that
proved the tables once and would not have noticed a JDK upgrade moving one.
22 mutants were run against the suite. 19 die. The three survivors are
equivalent today for a reason that is now a guarded invariant rather than a
coincidence: no script range reaches below U+0370, so the two Latin cuts
cannot differ; none of the 40 code points whose case the JDK disputes can
start a non-Latin run; and no empty line can be selected, so keeping trailing
empty fields on the newline split changes nothing. Each has a test that fails
if the enabling property does. The rounding mode needed the same treatment
the other way: it is unobservable through analyse at an even threshold, so a
half-up mutant passed the whole corpus, and the rule is now pinned by a
fixture and the unobservability itself asserted.
Also fixes a cliff rather than waiting for it: with every table in its field
initialiser the class stopped compiling with "code too large" once the
lowercase map was added, because array literals share one <clinit> and a JVM
method body may not exceed 65,535 bytes. Each table now has its own factory
method, and the generator refuses to emit one too large for that budget
instead of letting javac report it as a size error.
378 tests pass, including the end-to-end parity against the real graph.
`laya.presets` ships five question sets for the workflows laya is used for --
support triage, email triage, LLM guardrails, moderation, and model routing.
They are data, not code, and one changed word is a different question put to
the model and therefore a different answer. So they are compared against the
reference on what the model is actually shown -- the instruction, the rendered
options in order, the labels, the legend -- rather than on the Java object.
`stateField` comes with them, and it is the part a port gets wrong quietly. A
preset says which part of the state it reads by naming it in backticks ("What
does the customer want in `message`?") and this reads it back out, which is
what keeps the preset and the field name in step with no hand-maintained
table. Three details are load-bearing and now recorded:
* Python's \w, not Java's. Java's admits combining marks and join controls,
and `Character.isLetterOrDigit` excludes the underscore -- which alone
turns `customer_message` from one field into none.
* the scan resumes after the closing backtick, as findall does. In
"`a`b`c`" the fields are a and c; a scanner that resumed inside its own
match would offer b as well and turn a one-field preset into a
three-field one, which answers null.
* only the instruction is read. Criteria quote user-facing labels, so a
backtick in one is punctuation; scanning the whole rendered question
would report a field the question does not read.
The fixture grew the cases that pin all of that -- a spec naming nothing, one
naming two fields, one naming the same field twice, non-word backticks, and a
backtick confined to a criterion -- plus the preset that takes an argument.
That one holds a trap: `email_questions({})` returns the DEFAULT categories,
because the reference tests the argument for truthiness and an empty dict is
falsy. It reads like a way to ask for no categories and is not, and a port
that checked only for null would build a choice question with no options and
throw.
Two of the recorded hostile cases give an instruction of null or omit it.
They are not expressible here: `Question` refuses a blank instruction at
construction, earlier than the reference does. The test asserts that refusal
rather than skipping the case, so the difference is checked instead of
silently uncovered.
23 tests. 12 mutants were run against them -- a reworded instruction, a
reworded criterion, two options swapped, a score level reordered, a dropped
question, the noul sides swapped, an unordered criteria map, a shared
(cached) preset map, and each of the three stateField details above. All 12
die.
254 tests pass.
…uilt from
A route explains itself in a `reason` string, and that string reaches API
responses. Two of them are built with Python formats that Java spells
differently, and one of them interpolates the mixed segment -- a slice of the
caller's own text -- so neither difference is theoretical.
`%r` is CPython's repr. The quote is a single quote unless the string holds one
and no double quote; a backslash and the quote are escaped; newline, carriage
return and tab get letter escapes; and anything `str.isprintable()` calls
unprintable becomes a numeric escape in lowercase hex. That last rule is the
one an ASCII-only check misses, and it is not a corner: U+00A0 NO-BREAK SPACE
is unprintable to CPython and arrives in pasted text constantly, so stopping
at U+007F writes a raw control character into a response. U+00AD, U+200B,
U+FEFF, U+202E and the private-use and unassigned planes are the same story,
while every printable non-ASCII character -- CJK, emoji, a combining mark --
is deliberately left alone.
That needed one more recorded table, `str.isprintable()`, because the boundary
is 148,998 printable code points against 965,114 unprintable ones and it moves
with the Unicode version. It is gated the way the others are: a sha256 over
`repr` of every code point in Unicode, so the escape rule is proven
exhaustively rather than on the cases someone thought to list. The listed
cases are still there for the rules a reader can name -- all four quote
combinations, the letter escapes, the three widths of numeric escape.
`%.0f` rounds halves to EVEN. `String.format("%.0f", 12.5)` gives "13" where
CPython gives "12", and a share of one letter in eight is exactly 12.5%, so
the router would misreport a percentage it puts in front of a user. Also
reachable at 0.205 -> "20" and 0.625 -> "62". `Math.rint` is the right
primitive and needs no BigDecimal.
Both also carry an ASCII fast path for the new table, generated from the same
predicate as the table itself.
258 tests pass. The percent test additionally counts how many of its own cases
would come out differently under `String.format`, and fails if that count is
small -- otherwise it would be asserting that two agreeing implementations
agree.
Three checkpoints, and choosing between them is not a tuning question. The
English checkpoint does not degrade gently off English, it collapses: 0.100 on
20-option MASSIVE Hindi intent against 0.050 for random guessing, and it
reports high confidence while doing so (ECE 0.855). The multilingual
checkpoint gains 21 points on non-English XNLI and gives up about 6 on English
suites. So a wrong route is not a slightly worse answer, it is a confident
wrong one -- which is why every precedence level and every detection branch
has a recorded case here.
This commit is the decision and the registry. It runs no model and reads no
disk, so it is safe on every request and testable without a checkpoint; a test
pins that by routing through a Router pointed at paths that do not exist. The
load-and-evict lifecycle follows separately.
Precedence, highest first: an explicit model, an explicit task, a detected
typed-decisions workflow (opt-in only -- that checkpoint is fine-tuned on four
synthetic workflows and must not be a silent default), an explicit language, a
per-call hint, the router's installed hint, detection, then the default.
The `reason` strings are asserted, not just the model. They reach API
responses, and they are built from the two Python formats this port added last
commit: `repr`, and a `%.0f` that rounds halves to EVEN. Both see arbitrary
input, because one reason interpolates the mixed segment -- a slice of the
caller's own text. Two route cases exist only to pin that: seventy Latin
letters against ten Greek ones is a non-Latin share of exactly 10/80, so the
reason reads "12% of letters" and not "13%", and five accented characters in
exactly forty is the same halfway case on the other percentage. Without them
the router could misreport a percentage shown to a user and every other case
would still pass. A third carries a no-break space and a single quote inside
the mixed segment, so the reason has to switch to double quotes AND escape a
non-printable character in one string.
Two API notes. A hint is one functional interface rather than the reference's
"code or callable": `LanguageHint.of("pt")` for a fixed code, a lambda for a
language-identification model. Returning null abstains, and abstaining is a
real answer -- it falls through to detection instead of pinning a checkpoint
on no evidence, which is also what `LANG=C` in a minimal container gives.
And `Question.spec()` is new, because a schema signature has to serialise a
question the way the reference's dictionary holds it; its javadoc states the
one thing a typed question cannot carry, which is a caller's own extra keys or
an unusual key order inside `criteria`.
129 tests on the router, 388 in the suite. 18 mutants were run: 17 die. The
survivor is a finding rather than a gap -- the reference rewrites a task
spelled `typed_decisions` before resolving it, and that rewrite cannot change
an answer, because name resolution lowercases first and the alias table
already holds both spellings. The faithful branch is kept and a test proves
the redundancy, so it fails if either spelling ever leaves that table.
Also fixed a trap in my own test: a blank dynamic-test display name aborts the
whole factory, and an aborted container is not a failed test -- 38 recorded
cases silently never ran while the suite stayed green.
`load`, `lease`, `attach`, `preload`, `unload`, `loaded`, `predict` and
`close`, with an LRU cap of two resident checkpoints as in the reference.
Loading goes through a factory seam rather than a hard-coded directory layout:
the reference downloads from the hub, this port takes local paths, so the
mapping from a checkpoint to an Agent belongs to the caller --
`checkpointsRoot(path)` for the obvious layout, `agents(factory)` for anything
else. A seam for the same reason `InferenceSession` is one: the parts most
likely to be wrong here are which agent eviction picks, whether an attached
one is ever closed, and whether two threads asking for the same cold
checkpoint pay for two copies, and none of that has anything to do with ONNX.
Through a stub they are tested in milliseconds and run in every CI job instead
of only the parity lane.
ONE PLACE THIS CANNOT COPY THE REFERENCE, and it is why `Lease` exists. There,
eviction drops a reference and CPython's refcounting keeps the agent alive for
whoever still holds it. A JVM will not free a native ONNX session that way, so
the router has to close it -- and closing one that a caller is mid-prediction
with would crash that prediction. So eviction marks a slot retired and closes
it when the last lease is released; `predict` leases internally, so the
ordinary path is safe without a caller thinking about it. An agent handed in
through `attach` is never closed at all: the caller kept ownership.
Two bugs found while writing this, both in code I had just written:
* `acquire` registered one in-flight handle and then passed a different,
unregistered one to `build`. Every concurrent waiter would have blocked on
a latch nobody ever counted down -- a deadlock, not a duplicate build, so
no amount of counting loads would have surfaced it.
* a test asserted `"multilingual"` equals `"multilingual"`, which is true
whatever the router does. It now writes a checkpoint whose `pt`
temperature override makes the same question answer differently with and
without the language, asserts the two differ, and only then asserts the
router passed the one it detected.
20 lifecycle tests. 16 mutants, all 16 die, and two of them were worth the
trouble for what they revealed rather than for dying:
* removing the cache fast path in `acquire` passed everything. The fast path
is not an optimisation: it returns a resident checkpoint without touching
the build lock, so a request for a loaded model is not stalled behind a
cold load of a different one -- in a server, the difference between one
checkpoint loading and all traffic stopping. There is a test for that now.
* removing the in-flight latch also passed, because the build lock plus the
re-check already guarantee one build on the success path. The latch earns
its place on the FAILURE path: without it each waiter re-enters the
factory to discover the same failure, which for a load that fails by
timing out is six slow failures instead of one. The javadoc said the latch
was what prevented duplicate builds; that was wrong and now says what it
actually does.
The second of those also caught a flaw in the test harness: the stub failed a
build before waiting on its gate, so a failing build finished before a second
caller arrived and nothing was ever shared. The test was measuring the stub.
A real load that fails takes as long as one that succeeds, and the stub now
does too.
408 tests pass. The CI floors move with them, so a suite that aborts instead
of running still fails the lane.
Every option of a choice shares one `head_max_len`, so a seventeen-label set
leaves a few tokens per label and the model is shown truncated options.
Shortlisting embeds the state and each option, keeps the top k, and runs ONE
prediction over the reduced set. No second decision pass: the model still
scores every criterion it is handed, there are just fewer of them.
`Predictor` comes with it -- one interface, `predict(state, questions)`,
implemented by both `Agent` and `Router` with no change to either, since both
already had that exact signature. The reference accepts anything with
`predict` or `system_one`; this is the typed form of the same idea, and it is
what lets a caller shortlist identically whether the questions go to a fixed
checkpoint or a routed one. The .NET port has `ILayaPredictor` for the same
reason.
AN API GAP THE TESTS FOUND. My first cut took only a `Question`, and a
`Question` always carries a non-blank instruction -- while the reference takes
criteria and an OPTIONAL instruction separately, and its default is to embed
the state alone. So thirteen recorded rankings diverged, and not subtly: the
test had to invent a placeholder instruction, which changed every query text.
There is now a criteria overload with a nullable instruction, which is the
reference's actual signature.
WHAT CANNOT BE MATCHED BIT FOR BIT, measured rather than assumed. The
reference scores with `np.dot(matrix, vector)`, a BLAS matrix-vector product,
and how BLAS blocks that accumulation differs between implementations: on
Apple Accelerate one row of a four-row product differs in the last bit from
the same row computed alone. Every intermediate -- the dot, both norms, the
division -- agrees bit for bit between numpy and a sequential loop; it is the
blocking of the matrix form that does not.
That has a consequence beyond parity: recorded verbatim, these scores would
make the fixture disagree with ITSELF between a developer's macOS and CI's
OpenBLAS, and the drift gate would fail for a reason unrelated to any port. So
the fixture records twelve decimals -- four orders of magnitude tighter than
any porting error this repository has seen, four orders looser than the 1e-16
BLAS spread -- and regenerating twice now produces identical bytes. The LABEL
ORDER stays exact, which is the only thing a consumer observes and the thing a
ranking bug actually moves. A test guards the tolerance from widening: it
fails if the recorded precision and the comparison stop matching.
47 tests. 19 mutants, all 19 die -- but two needed real work rather than a
tick, and both were hidden by that very tolerance:
* removing the zero-norm query guard passed everything. A norm of zero does
not imply a dot product of zero once components underflow: a query of
1e-200 against a doc of 1.0 scores 1.4e-200 without the guard, which
agrees with 0 to twelve decimals. Asserted exactly now.
* removing the clamp to [-1, 1] passed too. The self-cosine of
[1/3, 1/3, 8/3] computes to 1.0000000000000002 with sequential
arithmetic, and that also agrees with 1.0 to twelve decimals. Asserted
exactly, both ends of the clamp.
Two of my own test bugs on the way: `subset_criteria` is a LIST when the
criteria were a list and I assumed a map, and the precision guard multiplied
by 1e12 -- which is not exact in binary, so it failed on its own arithmetic
rather than on the data. It reads the decimal scale through BigDecimal now.
`embed_fn_from_agent` is deliberately not ported: it mean-pools the
checkpoint's own encoder, which needs hidden states, and the ONNX graph this
runtime loads exposes logits. The reference says a dedicated bi-encoder
shortlists better than that helper anyway.
455 tests pass.
47 findings across port fidelity, Java defects, test vacuity and the
generators. 30 were real and are fixed here; 4 turned out faithful to the
reference and were deliberately left alone; 11 were my own tests asserting a
bug or unable to fail at all.
THE TWO THAT MATTERED MOST WERE IN CI, AND ONE OF THEM MADE A CLAIM IN THIS
PR UNTRUE. The drift gate ran `--check --only python_json.json`: one of six
model-free families. lang_detect, lang_tables, presets, router and shortlist
were gated nowhere, so a change to laya/ could move the contract those five
record while every check stayed green -- the Java tests assert against the
committed copy, which is exactly the failure the fixtures exist to prevent. It
is a bare `--check` now, and it immediately caught real drift in three
families. And `--only` with a typo exited 0 printing "fixtures match laya", so
one hyphen for an underscore turned the whole gate into a no-op that asserted
the opposite; an unknown family name now exits 2, and the success path refuses
to report a match when it examined nothing.
Also removed four hardcoded copies of a developer's home directory as the
default checkpoint root. Besides being a leaked path, it made WHETHER a family
regenerates a property of whose machine ran the script, and wrote that path
into a committed skip marker.
Behavioural defects, worst first:
* a NaN cosine ranked FIRST where numpy's argsort puts it LAST. Double.compare
treats NaN as the largest double, so a descending comparator inverted it --
and that does not merely misreport a score, it changes which labels survive
the cut, so the model is asked a different question.
* a throwing Agent.close() during eviction aborted the build after the new
slot was published, stranding its lease: nothing could release it, so the
agent could never be retired or closed. An unreachable, unclosable native
session, and every waiter told the load failed when it had not.
* Lease.close() was documented idempotent but raced itself. Two threads
closing the same lease both passed the check, both decremented, and the
count hit zero while another lease was open -- closing an agent under a
caller still using it, the single failure the mechanism exists to prevent.
* eviction closed agents while holding the registry lock, stalling callers
asking for an already-resident checkpoint by as long as the close took.
Measured at 1.3 s. That is precisely what the fast path exists to avoid.
* attach() closed the very agent it is told the caller owns, when the router
already held that instance -- which is the case its own javadoc invites.
* String.trim() where the reference uses str.strip(), in normaliseName and
englishFromCode. Python's strip treats U+00A0, U+0085, U+2007 and U+202F as
whitespace and Java's trim does not, so "en" plus an ideographic space --
what a CJK input method produces -- routed to the MULTILINGUAL checkpoint
while the reason string still said the caller asked for English. The same
gap sat in Question's label handling and instruction check, where
UnicodeTables.strip already existed and simply was not used.
* the embedding cache had no dimension guard, so where the reference refuses
a changed embedder and keeps a clean cache, this accepted it and poisoned
the cache permanently; counted misses per distinct text where the reference
counts per occurrence, breaking hits+misses == texts looked up; and handed
out its own row arrays, so a caller's write corrupted every later lookup.
* a null question id aborted the whole route via Set.of().containsAll, where
the reference merely fails to match and carries on.
* percent0 spelled a non-finite as "NaN"/"Infinity" where CPython's %.0f
gives "nan"/"inf". Latent -- both call sites are finite by construction.
Left alone on purpose, because the reference does the same: maxLoaded is a
high-water mark; analyse() reports script and rates from the joined window
while taking the verdict from a leaf, which can emit "0% non-English letters;
not safe for the English checkpoint"; task rejects workflow names with a
message about models; cache misses are counted before the embed is attempted.
Each is now documented as deliberate rather than looking like an oversight.
On the test side: five @testfactory bodies generated zero assertions when
their fixture key emptied, and the suite exited 0 -- they have floors now,
each naming the row that has to exist. Two score loops had no length
assertion. A hostile-case test asserted the throw on the first question in a
spec rather than the one missing its instruction. One assertEquals had its
arguments reversed. And four regression tests now cover the lifecycle defects
above, including the close race and the stranded lease -- both of which my own
new tests caught on the first run, before the fixes were complete.
480 tests pass. Every generator gate is clean and regenerating twice gives
identical bytes.
Two more adversarial passes, this time over the code that merged in NandhaKishorM#927 and had only ever had mutation testing and parity fixtures. 25 findings; the release-blocking ones first, because any of them could have done damage that cannot be undone. RELEASE AND CI * release-java.yml on a manual run from a BRANCH published to Maven Central with the branch name as the version. GITHUB_REF_NAME is the branch there, so the version became `main`; the snapshot guard only matched `*-SNAPSHOT` and let it through; and the build job's version check was guarded by a tag condition the publish job did not share, so the two jobs disagreed about what was being released and the artifact published was never the one the tests ran against. Central is append-only: `:main` could never have been replaced. It now refuses to publish from anything but a tag. * the release gate's test floor was 80 against a 484-test model-free suite, so 83% of the tests could vanish and the publish still went ahead. Now 440, matching the build lane. * check_test_results.py compared the floor against JUnit's `tests` attribute, which COUNTS SKIPPED TESTS. A suite in which every single test aborted satisfied a floor of 440 and both --allow-aborted lanes -- the build gate and the Central release gate -- went green with nothing executed. That is the exact failure the script's docstring says it exists to catch. It counts executed tests now, and an all-skipped suite fails. * the empty `laya-java-client` module was in the Gradle build with a full MavenPublication, so a release would have pushed a zero-class jar to Central with a POM promising an HTTP client the README itself lists as not implemented. It joins the build when it has code. * publishing required two of the four secrets. Without the signing passphrase the build calls useInMemoryPgpKeys(key, null) and uploads UNSIGNED artifacts, which Central rejects only after the upload. THE SPLIT GRAPH LAYOUT WAS DEAD. `encoder.onnx` + `head.onnx` is a documented, advertised layout, and the head declares `qtype` as [B, 1] while this passed a rank-1 tensor -- so ORT refused every graph the only exporter for that layout produces, and the error surfaced as a per-batch backend failure that reads like a bad model rather than a bad client. laya-ts passes [n, 1] for the same reason. OTHER RUNTIME DEFECTS * a temperature written as a JSON string was discarded and silently defaulted to 1.0, where the reference parses it. Measured: the published answer_confidence -- the number callers gate on -- was overstated by 19 points with no warning, in a class whose whole stated purpose is that these are read from the checkpoint and never defaulted. * a temperature of the wrong TYPE (a number, a string, a mapping) loaded cleanly at 1.0 where the reference refuses the checkpoint. The wrong LENGTH was already refused; only the type slipped through. * a logits output of the wrong rank or dtype escaped as a raw ClassCastException naming neither the graph nor the output, through a method whose javadoc promises the backend never leaks -- so a caller's catch (InferenceException) did not catch it. An fp16 or fp64 re-export reaches it. * run() never consulted `closed`, so the guard isClosed()'s own javadoc promises ("a caller gets an API error, not a backend one") did not exist, and a use-after-close produced exactly the error class the API says it never emits. * the action softmax padded a one-column head up to two columns, turning a probability of 1.0 into a sigmoid -- a plausible-looking number for a value the model never produced, in the field callers gate escalation on -- and published NaN for a zero-column row. It uses the graph's own width. * round4 lost the sign of a negative zero, so the exact-parity claim did not hold and a record's equals() would have reported two equal answers as different. THE JSON LAYER, which parses every fixture and every tokenizer.json * no depth limit in either direction: deeply nested input threw StackOverflowError -- an Error, which a server's catch (Exception) does not contain -- where CPython raises a catchable RecursionError. The identity-set fix had closed only the cyclic case. Both now have one. * an integer literal was converted with a QUADRATIC BigDecimal call and no digit cap: a 1 MB literal cost 21 seconds of a request thread, from one field of a checkpoint downloaded off the hub. Capped at CPython's 4,300. * a backslash-u escape accepted fullwidth and Arabic-Indic digits as hex, because Character.digit is Unicode-aware, so two byte-different documents parsed to the same value. ASCII only now. 481 tests pass, every generator gate is clean, and one test that had been asserting the temperature defect was corrected rather than the fix weakened.
…s Unicode, not the reference's
The only live defect of the whole review: it fires on the shipped english
checkpoint, with no config change, and the same jar gave different ids on a
different JDK.
The GPT-2 pre-tokenizer pattern is
's|'t|'re|'ve|'m|'ll|'d| ?\p{L}+| ?\p{N}+| ?[^\s\p{L}\p{N}]+|\s+(?!\S)|\s+
and `\p{L}`/`\p{N}` in java.util.regex follow the JDK's Unicode version.
Corretto 17 ships Unicode 13.0; the reference tokenizer classifies Unicode
15.1 and 16.0 additions as letters. They disagree on 9,917 code points -- CJK
Ext-I, Egyptian Hieroglyphs Ext-A, Arabic Extended-B, Vithkuqi, Kawi, Tangsa,
Garay, Todhri, Tulu-Tigalari and others, plus several scripts' digits.
For every one of those the seven contraction tokens could never form: when
` ?\p{L}+` fails, ` ?[^\s\p{L}\p{N}]+` swallows the apostrophe into the same
piece, so `'s` `'t` `'re` `'ve` `'m` `'ll` `'d` are never produced. Measured:
99,170 of 119,004 probe inputs diverged, including the ordinary frame "The X's
value". Three Unicode versions are in play and no two agree -- the JDK's
13.0, CPython's 15.0 (which this repo's other tables record), and the
reference's -- so the tables already here would not have fixed it either.
The classes are therefore recorded FROM THE REFERENCE and compiled in, which
is what every other table in this port does. They cannot be read as a
property, so the probe is behavioural and follows from the pattern itself: a
code point is a letter if "<cp>'s" leaves "'s" as its own piece, and a number
if "1<cp>" stays one piece. A full sweep of all 1,114,112 code points takes
about five seconds, and `--check` gates the committed copy in the parity lane,
which is where a checkpoint is available.
The pattern is now a scanner over those tables rather than a regex, in the
same style and for the same reason as the language detector's four scanners.
The alternation order is the specification and is preserved, including the
`\s+(?!\S)` backtrack that leaves a whitespace run's last space to the
following word -- which is what produces "Ġthere" rather than a lone
whitespace piece. Whitespace stays `Unicode.isWhiteSpace`, which already
matches Rust's `char::is_whitespace` on every code point; that is what
UNICODE_CHARACTER_CLASS was buying and the only part of the flag that worked.
Verified against the reference on both shipped checkpoints: 216 targeted cases
(all seven contractions against the code points that used to diverge) and a
34,395-case sweep striding every plane in three frames -- 0 divergent, where
the regex diverged on 99,170 of 119,004.
481 tests pass.
The reference's rule is `truncate_left = isinstance(state, list)`, and this
passed a literal `false`. A conversation is serialised newest-last, so an
oversized history was cut from the right: the model was shown the OPENING of
the conversation and answered about that, while the current turn -- the thing
the question is about -- was the part thrown away.
Measured on a 120-turn history: the reference's window started mid-message-95
and ended at turn 119; this one started at turn 0 and ended at turn 24. End to
end on the real fused graph, one option's probability moved by 3.8x (0.0364 to
0.0096) and the score moved 2.0685 to 2.0265. And `usage` was byte-identical
between the two -- same input_tokens, same state_tokens_dropped, same
truncated flag, same truncated_questions -- so nothing a caller could read
revealed it. Left-truncation is the entire reason that flag exists, and the
only production caller never set it.
Three smaller parity defects alongside it, all found the same way:
* a null state was answered as a decision about the literal text "null",
byte-identically to passing the string, at full confidence. Both Python
backends raise `TypeError("state must not be None; ...")` and the
reference's own comment gives that serialisation as the REASON to refuse.
The comment here claimed the opposite -- that a null state is legitimate
"because Python serialises None to the text null" -- which would have
stopped the next reader from fixing it. So would the test, which was
titled "a null state is answered, as Python answers it".
* an empty questions map threw, where the reference returns empty answers
and zero usage per state with no tokenization and no forward pass. That
meant the two runtimes could not be swapped under a question set derived
from a filter, where an empty schema or a fully disabled rule set is a
well-formed request with a well-formed empty answer. The test asserted the
throw.
* a checkpoint naming no pad_token silently fell back to the SEP id, where
the reference fails the request. The graph masks padding so the numbers
did not move; the divergence was that a mis-exported checkpoint failed
loudly on one runtime and answered quietly on the other.
Two of the three had a test pinning the wrong behaviour, and one of those
asserted a false claim about the reference in its own display name. Both are
corrected rather than deleted, and the left-truncation fix has a regression
test that decodes the surviving window and asserts the newest turn is in it
and the oldest is not.
Worth recording what the same review found CLEAN, because it is the part that
was most at risk: the head budget and option truncation over 1,483 cases
compared id-for-id (including 506 re-capped, 400 option-collapse, 72
marker-dropping and 337 truncating cases); row attribution across 37 uneven
states x 4 questions x 10 batch and sort configurations, 370 of 370 identical;
the collator's padding and masks over 192 collations, element for element; and
every usage field including which question is blamed for a truncation. Those
are the places an off-by-one silently returns another state's answer, and they
hold.
482 tests pass.
The README still described the SDK as prediction-only. Add the four sections for the features this branch adds, and correct the status list so it no longer claims the router and detection are missing. The "lower-level pieces" snippet was also missing three imports it uses (AgentConfig, Tokenizer, Path), so it could not be pasted as written. Every java block in the file now typechecks against the compiled classes.
…question
The probe behind PreTokenizerTables asked whether "<cp>'s" leaves "'s" as its
own piece. That is not a test of class membership: it is true of every code
point that fails to ABSORB the apostrophe, which includes every whitespace
code point and every digit. 1,935 code points were recorded as letters that
are not -- all 1,911 digits and the 24 non-space whitespace characters -- and
LETTER and NUMBER overlapped on all 1,911.
Because ` ?\p{L}+` is tried before `\s+`, a letter run then swallowed inner
whitespace: "a b\t\tc\n\nd\r\ne f" encoded to 11 ids where the reference
gives 14, the tabs and newlines merging into the surrounding word instead of
becoming their own pieces. "abc123" collapsed to one piece where the reference
gives two. Both fire on the shipped english checkpoint with no config change.
The multilingual checkpoint is unaffected -- its Metaspace pre-tokenizer does
not use these classes -- which is why only one corpus entry failed.
The probe is now anchored by a character of the class being tested: a code
point is a letter if "a<cp>" stays ONE piece and a number if "1<cp>" stays
one. The only way the first match can reach past the anchor is the alternative
under test consuming the code point, so one piece means membership and two
means non-membership, with no third reading. The misleading comment claiming
the old judge was written so "only a genuine class membership answers true"
is replaced by what makes the new one sound.
Worth being plain about how this got in: the sweep quoted in 0e1f1b1 was 216
targeted contraction cases and a 34,395-case stride sweep in three frames,
none of which put whitespace or a digit BETWEEN two letters -- the one shape
that makes an over-inclusive letter class observable. The corpus fixture
caught it, and only with a checkpoint attached. The sweep now covers 115,699
cases including whitespace and digit frames in twelve positions: 0 divergent
on both shipped checkpoints, where the old table diverged on 32 of them on the
english checkpoint and on none on multilingual.
PreTokenizerTablesTest pins it model-free, because regenerating cannot catch a
bad probe -- the generator agrees with itself whatever it asks. It asserts the
properties the pattern's alternatives imply rather than the table's contents:
no whitespace code point is in either class, the two classes are disjoint, an
inner whitespace run is cut exactly as the reference cuts it, and a digit run
stays its own piece beside a word. Four of the five fail against the old
table. The fifth guards the Unicode skew 0e1f1b1 fixed, over 751 code points
across five blocks that are letters to the reference and not to Corretto 17.
487 model-free tests pass, 634 with the checkpoints and the fused graph
attached, and every generator gate is clean.
The java lane had two jobs where the .NET lane has five, and neither of the
missing three was a cosmetic gap.
A JDK MATRIX, which is the one this port actually needed. `\p{L}` and `\p{N}`
in java.util.regex follow the JDK's own Unicode version, and that is how the
pre-tokenizer came to return different token ids from the same jar on
different JDKs. Measured: `Character.isLetter` answers 0 of 751 on Corretto 17
(Unicode 13.0) and 751 of 751 on Corretto 24 (Unicode 16.0), across the five
blocks this port has to classify.
A lane that merely installs another JDK would not have caught it, because the
toolchain pins the COMPILER to 17: Gradle would compile and test on 17 whatever
setup-java installed, and the cell would go green having tested the same thing
twice. So `-PtestJavaVersion` moves the test JVM only; `options.release` keeps
the bytecode at 17, verified as class file major 61 while the tests ran on
24.0.2. Two guards stop it being vacuous. `launcherFor` with auto-download off
fails the build outright on a JDK the runner does not have -- "Cannot find a
Java installation on your machine matching: {languageVersion=19}" -- instead of
falling back to 17; and TestJvmVersionTest asserts the running JVM from inside
the suite, because Gradle writes `<properties/>` EMPTY into the JUnit XML, so
neither the report nor any later step can tell 17 from 24. Forcing the launcher
to 17 while the lane asks for 24 fails that test.
The model-free suite is the right scope for the matrix: the JDK-sensitive tests
are exactly the ones needing no model -- the character tables behind a sha256
over all 1,114,112 code points, the White_Space predicate, the pre-tokenizer's
letter and number classes, CPython's float repr and the rounding rules. All are
compiled in FROM the reference so the host JDK's Unicode cannot move them, and
this is what measures that. 635 tests pass on 17 and 24 alike, 0 skipped.
THE PARITY LANE IS NOW TWO CELLS, split by whether the cell needs the traced
graph. Only one of the four checkpoint-backed test classes does; the other
three need `tokenizer.json` and `rl_agent_config.json`, which are kilobytes.
Measured: the tokenizer cell is 183 tests in 8 seconds against 3 fixture
families regenerated in 13, and it was waiting on a gigabyte-scale export it
never reads. That export is the whole runtime of this lane, and it is also
where the pre-tokenizer bug was caught -- a check that cheap should not take 75
minutes to report.
NOT split per checkpoint, which is where the .NET lane splits, and the reason
is worth recording so nobody "fixes" it later. The fixtures are not
per-checkpoint documents: tokenizer_ids.json, sequences.json and decode.json
each carry BOTH checkpoints' sections in one file, so a job holding one of them
regenerates none of the three and --strict fails by construction. And the Java
tests assert that both checkpoints' cases are present -- a floor that exists
because a @testfactory whose fixture key emptied used to generate zero
assertions and still exit 0. Splitting per checkpoint would mean deleting that
floor to buy nothing, because there is no export to parallelise away. Verified
by hand: with one checkpoint present the suite reports a FAILED container, not
a skip.
Smaller things the split turned up:
* a blank env var is not an absent one. A matrix cell that owns no graph
writes `LAYA_ONNX_GRAPH: ''`, because an expression yielding the empty
string still DEFINES the variable, and the build passed it straight
through -- so the graph-backed factories would have failed on an
unopenable path where they are written to abort by assumption. Blank now
counts as absent, and with it the suite reports identically either way:
628 tests, 3 skipped, both for unset and for empty.
* the floors are per cell now, 175 and 9 against 183 and 9 measured. Dropping
three of the tokenizer cell's four classes leaves Gradle reporting BUILD
SUCCESSFUL on 5 tests, and the floor is the only thing that catches it.
* `--tests` patterns are fully-qualified class names, not `*Glob`. An
unquoted `*Foo` interpolated into a `run:` line is a bash glob that
survives only while nothing in the working directory matches it.
* the hyphenated matrix key is read as `matrix['min-tests']`; by dot access
an expression parser can read the hyphen as subtraction.
* the new job runs check_test_results.py, so it sets Python up like the other
two rather than trusting the runner image to expose `python`.
* setup-java's variable suffix follows `runner.arch` rather than a hardcoded
`_X64`.
The lane stays advisory, like the .NET one and for the same reason: the Python
package is the product, and a Python-side improvement must not be blocked on a
port catching up.
Six findings against the lanes added in the previous commit. Three of them made
a cell green while measuring the wrong thing, which is the failure a CI change
is most able to hide.
THE JDK FLAGS WERE SILENTLY IGNORED. Gradle does not read
`org.gradle.java.installations.*` from `-D`; it must be `-P` or
gradle.properties. Measured on this build with the `javaToolchains` report:
`auto-detect=false` passed as `-D` left all 8 locally detected toolchains in
place, and as `-P` dropped it to 3. So `fromEnv` never registered setup-java's
tool-cache installs and `auto-download=false` never applied -- and the comment
claiming those two made a misconfiguration loud described something that was
not running. The cell passed anyway, by coincidence: setup-java makes the LAST
`java-version` the JAVA_HOME, so Gradle's "current JVM" supplier happened to
supply the right launcher while the image's own JDK 17 served the compiler.
That is precisely the fragility naming the variables was supposed to remove.
Now `-P`, with `auto-detect=false` added so the only installations are the two
named. Verified both directions: asking for 24 with auto-detect off and no
`fromEnv` exits 1 with "Cannot find a Java installation on your machine
matching: {languageVersion=24}"; adding `fromEnv` exits 0.
`--allow-aborted` FORGAVE THE ONE TEST THAT PROVES THE JDK LANE IS NOT VACUOUS.
TestJvmVersionTest is `@EnabledIfSystemProperty`, and the build sets that
property only when `-PtestJavaVersion` is present. If the flag ever stops
reaching the build -- renamed on one side, lost from a folded `run:` block,
overridden in gradle.properties -- then no launcher override happens, both
cells run on the compile JDK, and the single test that would notice is SKIPPED
rather than failed. This lane needs `--allow-aborted`, because the
checkpoint-backed factories abort here, and that flag permits exactly that
skip. Measured with the flag dropped: Gradle exits 0, the floor exits 0 on 488
tests, and the lane reports two JDKs having run one of them twice.
`check_test_results.py` grew `--require-passed`, which demands a substring
match against a test that actually PASSED rather than merely registered; with
it, the same run exits 1 and names why ("it was skipped").
THE EXPORTED-GRAPH CACHE KEY DID NOT INCLUDE THE PINNED REVISION. HF_REVISION
lives in prepare_checkpoint.py, which was not in the key's `hashFiles`, and the
script short-circuits on an existing graph. So bumping the revision changed
nothing the key could see: the graph traced from the PREVIOUS revision was
restored and reused, `predict.json` was re-recorded from that same stale graph,
and the two sides agreed. Nine tests pass, cell green, having never measured
the pinned checkpoint -- with the informational drift step showing
`predict.json` unchanged, which reads as reassurance. The sibling lane does not
have this bug; its key hashes the file its own revision lives in.
Fixed at the source rather than only in the key: prepare_checkpoint.py now
stamps the revision beside what it produces and reuses a checkpoint or a graph
only when the stamp matches, so a restored cache from before a bump is
re-downloaded and re-exported instead of trusted. The stamp also records
whether weights were fetched, because a weightless checkpoint cannot be traced
and must not satisfy a request that needs one. The key additionally carries the
cell name and prepare_checkpoint.py as defence in depth.
THE HUGGING FACE CACHE WAS A PERMANENT NO-OP, and is removed rather than left
looking like a cost that was being avoided. prepare_checkpoint.py passes
`local_dir` to `snapshot_download`, which writes the files there and not into
the hub cache. Measured against the pinned huggingface_hub: with a revision
pinned, as here, `~/.cache/huggingface/hub` gets nothing at all; without one it
gets a `refs/main` pointer and zero blobs. The step therefore always missed on
restore and saved nothing, every run paid the full download, and its key also
lacked the cell name -- so the moment it was made to work, the two cells would
have raced for one key while holding different content and the weight-bearing
cell could have restored a weightless tree. What making it real would take is
recorded where the step was.
Two smaller ones:
* the test floors were 440 against suites that execute 487 and 488, so a
third of either could vanish unnoticed -- and that floor is the only thing
between a @testfactory that stopped registering cases and a green cell.
Raised to 480. NOT to the ~620 a full run reaches: that is a
checkpoint-bearing total, and a model-free lane would fail it on the first
run.
* `matrix['min-tests']` is back to `matrix.min-tests`. The bracket form was
defensive about a hyphen that needs no defending -- actionlint types the
key and lints the dot form clean, and the .NET lane already spells it that
way. Two lanes should not spell the same thing two ways.
All five lanes verified against their exact commands: 488 executed in the JDK
cell with TestJvmVersionTest passing, 487 in the build lane, 183 in the
tokenizer cell, 9 in the predict cell. actionlint with shellcheck is clean on
every workflow in the repository except docker.yml, which is untouched here and
has a pre-existing SC1007.
Groundwork for the email port, as its own commit because the tables are gated
exhaustively and the port is not written yet.
`laya.email` reads two character properties this port did not record, and both
were nearly implemented by reusing a table that already existed. Both reuses
would have been wrong, on large and specific sets, so the numbers are asserted
rather than left to a comment.
MARK is the Mn/Mc/Me categories -- `unicodedata.category(ch).startswith("M")`
in `_drop_marks`, `\p{M}` in the TS port. COMBINING is not a substitute: it is
the canonical combining CLASS, and 1,528 code points are category M with a
class of zero -- U+034F COMBINING GRAPHEME JOINER, the Cyrillic Me numerals,
the Thaana vowels. Marks are dropped before a sign-off's tail is matched
structurally, so using the class would leave those in and a name carrying one
would stop reading as a name.
INITIAL is the Lu/Lt/Lo categories, the test `_is_english_signoff` applies to
each token's first letter. UPPER is not a substitute either, and not merely
imprecise: `str.isupper()` is the Uppercase property, true of 1,951 code points
against these 133,474, and the two sets are not nested in either direction.
131,643 initials are not uppercase -- every caseless script, so "Obrigado,
山田" would stop being a sign-off -- and 120 uppercase code points are not
initials, the circled and squared letters of category So, so "Thanks, Ⓐ" would
start being one.
Recorded from CPython 15.0.0 like every other table here, with an exhaustive
sha256 over all 1,114,112 code points in the lang_detect fixture, and a third
test pinning the three counts above plus the five code points that make the
difference observable. Delegating isMark to COMBINING or isInitial to UPPER
fails four assertions each.
638 tests pass, and every generator gate is clean.
The cleaner and the state builder, matching the reference byte for byte on 93
recorded cases and on a 40,000-case differential sweep: 0 divergent.
WHAT HAD TO BE TRANSLATED RATHER THAN TRANSCRIBED. Four of Python's regex
constructs mean something else in java.util.regex, and three of them are the
kind that are right on the cases you thought of:
* `\s`. Python's is 29 code points, exactly `str.isspace()` -- verified over
all of Unicode. Java's is 6 by default and 25 under
UNICODE_CHARACTER_CLASS, which is the White_Space property and therefore
misses U+001C to U+001F; those four are Cc, and Python calls them space.
* `\d`. Python's is Nd, 680 code points. Java's is ASCII by default and,
under UNICODE_CHARACTER_CLASS, follows THE JDK'S Unicode version -- the
defect this port shipped in its pre-tokenizer, where the same jar gave
different token ids on different JDKs. A date written in Arabic-Indic or
Devanagari digits is a date, and an ASCII-only port keeps the quoted
history instead.
* `.` and `$`. Python's `.` excludes only `\n`; Java's excludes every line
terminator, U+0085 and U+2028 among them, and its `$` looks for any of
them. Every pattern carries UNIX_LINES so both mean what Python means.
* `\b`. Below.
`\s`, `\S` and `\d` are therefore built FROM the recorded tables rather than
written out, so they and the rest of the port read one Unicode and a table
change moves both together.
THE SIGN-OFF RULE is a scanner, as `laya.lang`'s four are and for the same
reason: its classes are Python's. `[^\W\d_]` opens a token and `\w` continues
it, and Java's `\w` is a different set -- it admits the combining marks the
reference has just dropped and the join controls that must keep separating
tokens. Each token's first letter must be Lu, Lt or Lo, which is what the two
new tables are for: `str.isupper()` would lose every caseless script
("Regards, 山田" is a sign-off) and gain the circled letters of category So
("Thanks, Ⓐ" is not).
`\b` COST 220 TIMES WHAT IT WAS WORTH, and the way out is the interesting part.
A word boundary is defined in terms of `\w`, so the exact translation is a
lookaround over Python's class -- which inlines 10,622 characters at 26
boundary sites, makes a 277,309-character pattern, and runs at 1,742 us per
paragraph against 7.8 us for Java's own `\b`. It was also the whole cost of
cleaning a message: 2.2 ms per case, against Python's 0.026 ms.
The two forms agreed on all 20,000 paragraphs of the sweep, and that is exactly
why the lookaround was not kept as the safe option: agreement on a corpus is
not equivalence, and Java's `\w` is the JDK's. So the boundary is still decided
by the recorded table, but outside the pattern -- `boundaryView` rewrites the
input so that Java's `\w` and Python's agree on every character of it, and
Java's fast `\b` then finds Python's boundaries. Both directions of the
disagreement are real and both are exercised: a mark or join control is Java's
word character and not Python's, so it becomes U+0000; U+00BD and the other
other-numbers are Python's and not Java's, so they become '0'. Both substitutes
are one character wide, are matched by the `[^.]` windows the patterns measure,
and appear in no literal of any pattern here -- which is asserted, not argued,
because it is the whole reason a substitution is allowed to be invisible. The
corpus rewrites 12,583 of 40,000 bodies, so neither branch is dead.
Three more measurements, each made before the change it justifies:
* a necessary-condition filter in front of the disclaimer pattern, since
every branch of it requires one of 24 literals. 2.2 ms to 0.99 ms. It is a
filter and never a decision, so a false positive costs only time -- and it
is matched with the pattern's own flags rather than by lowercasing, because
`re.I` folds more than lowercasing does and a filter that missed U+017F
would reject a disclaimer the pattern accepts.
* `javaWord` spelled out from the definition the JDK documents for `\w`,
rather than asked of a matcher per character. Compared against that
matcher over all 1,114,112 code points: identical.
* the character classes built in one walk of Unicode instead of three, and
the unused word class removed from the holder. First call into the class:
160 ms to 52 ms. The classes are byte-identical to the three-walk version,
which is what makes the refactor a refactor.
End to end: 0.029 ms per case against the reference's 0.026, warm and best of
three on the same 4,000 bodies -- from 85 times slower to parity, with the
exactness unchanged at every step. The patterns are built on first use, so a
caller who only ever opens an Agent pays none of it.
WHAT THE CORPUS IS. 93 cases, and the near-misses outnumber the hits, because
the module argues in four separate places that a cleaner which is too eager
deletes the sender's request. `From: my side the integration works` is prose,
`Em resposta ao que você escreveu:` carries no date, `De: 10/09 a 15/09` is a
leave request, `Thanks for the quick reply.` is a sentence, `Obrigado pelo
retorno, mas ...` is a request, and `Is this confidential?` is a question.
Ten mutants were written against the port and SIX SURVIVED the first corpus --
the combining-class table, the mark-without-a-base rule, the 40-character
sign-off limit, the three-token tail, the attribution head, UNIX_LINES and both
space classes were all invisible to it. Eight cases were added to kill them, a
non-ASCII-digit case for a seventh, and the thirteen mutants are now all killed.
That is the second time in this branch that a green fixture meant nothing until
a mutant proved it could fail.
742 tests pass, identically on JDK 17 and JDK 24, every generator gate is
clean, regenerating email.json twice gives identical bytes, and all ten of the
README's java blocks compile against the built classes.
…rt added 594 model-free tests execute now, against floors of 585. They were 480, which was the same slack the review of these lanes flagged when the suite was 487 -- a floor that drifts behind its suite stops being a gate, and this one is the only thing between a @testfactory that quietly stopped registering cases and a green cell. Measured, not computed: `./gradlew test` with no checkpoints reports 599 found and 5 skipped, and the five skips are the three checkpoint-backed factories plus the JDK-version guard, each of which is a deliberate abort in this lane.
The 40,000-case sweep that cleared the port had NO astral character in it. A
20,000-case corpus of them -- marks, letter- and other-numbers, mathematical
letters, CJK extensions, emoji, at every offset of a disclaimer and at every
width around the sign-off limit -- diverged on 4,609. The units the reference
counts are code points and the units Java reaches for are chars, and that is
three of the four; the fourth is mine.
THE BOUNDARY VIEW WAS RECONCILING AGAINST A CLASS NOTHING CONSULTED. The
previous commit replaced the exact `\b` lookaround with Java's `\b` applied to
a string rewritten so that Java's word class and Python's agree on it. Java's
`\b` only reads the Unicode word class when UNICODE_CHARACTER_CLASS is set, and
it was not, so `\b` was reading `[a-zA-Z0-9_]` while the view was built against
the Unicode one. The symptom is a false positive, which is the direction that
costs a user something: "Esta mensagem" followed by a letter-number has no word
boundary to Python, an ASCII `\b` finds one anyway, and a sentence the reference
keeps was deleted whole. The flag is set on that one pattern now; it has no
`\s`, `\d` or `\w` of its own -- those are spelled out from the tables -- so the
flag reaches nothing else.
The other three are counting:
* the final cut. `text[:max_chars]` is code points; `substring(0, maxChars)`
is chars and splits a surrogate pair, returning half a character the
reference cannot produce. 4,641 of the 4,609 divergent cases are this one.
* the pre-truncation bound, `max_chars * 4`, the same way. It decides whether
a disclaimer is inside the window at all, so cutting it short lets a
truncated footer that no longer matches survive into the model's input. The
final cut hides this for most inputs, which is why the case that pins it is
specific: three astral characters and a budget of 11.
* the signature width. `len(line.strip()) <= 40` counts code points, so a
40-code-point closing is 41 chars wide with one astral letter in it and the
signature stays.
Nine cases are added to the corpus, one per defect plus the near-misses that
keep each fix from being "never match": a disclaimer whose astral character is
somewhere harmless still matches, and a sign-off one code point over the limit
still survives. All four mutants die against them, and the first version of the
pre-truncation case did NOT kill its mutant -- the final cut masked it -- so the
case in the corpus is the one found by searching for a discriminating
(padding, budget) pair rather than the one I wrote first.
Both sweeps are clean: 0 of 20,000 astral and 0 of 40,000 on the original
corpus, which is the regression check for fixes aimed at the first. 751 tests
pass, identically on JDK 17 and JDK 24.
Worth saying plainly, because it is the second time on this branch: the port
had already been through 93 recorded cases, a 40,000-case differential sweep
and thirteen mutants, and all of that passed while four defects sat in it. A
corpus only tests what it contains, and mine contained no character above
U+FFFF.
Twenty-two findings. One is a runtime defect, four are tests that could not
fail, and the rest are claims in comments that were not true.
A HOSTILE LINE COST 4.8 SECONDS. `ATTRIBUTION_TAIL` is
`^.{0,120}\S@\S+\s+(wrote|...)\s*:\s*$`, and `.{0,120}` offers 121 starting
points; at each one a greedy `\S+` ran to the end of the line and gave every
character back, because the `\s+` after it can never match where a `\S` was
just surrendered -- the classes are complements. 121 full scans of the line.
Measured on a 4,000,000-character line: 55 ms at the default budget and 4.8 s
at a `maxChars` of 300,000, against 0.018 ms for an ordinary message -- and
`emailState`'s own javadoc tells callers that raising `maxChars` is worth it.
Two changes: `\S++` possessive, which forbids backtracking that could never
have succeeded, and a necessary condition in front of the pattern, since a tail
that matches is a line whose rstrip ends with a colon. 55 ms to 9 ms, 4.8 s to
66 ms, both sweeps still clean.
FOUR TESTS COULD NOT FAIL, and the filter they were guarding had five dead
literals because of it.
* the disclaimer filter's necessity test checked thirteen examples against
the WHOLE filter. Every example was satisfied by some literal, so a literal
that only one branch needs was invisible: `sigilos`, `exclusivamente`,
`uniquement`, `recebido` and `receber` could each be deleted with the suite
green, and "Esta mensagem e sigilosa." then reached the model. The filter is
now twelve literals, each the shortest form covering its branch, and the
test gives each one a disclaimer whose ONLY hint literal is that one and
asserts that removing it stops the filter matching. All five previously
deletable literals now fail. (`privilegiad` really was redundant --
`privil[eé]gi` covers it -- so it is gone rather than tested.)
* three of the pattern's twelve alternations were never the sole matcher, so
breaking them killed nothing. The per-literal table covers each.
* the @testfactory floors were counts well under the corpus: 70 against 86,
5 against 9. Trimmed to those floors the suite stayed green while losing
every astral case, and the three mutants the full corpus kills -- the
code-point width, the surrogate-safe cut, the `\b` flag -- all survived.
The corpus is now checked by NAME: thirty cases are listed as load-bearing,
and the sizes are exact equalities to be changed knowingly.
* the signature window `max(1, min(int(len * 0.6), len - 8))` was entirely
unpinned: the ratio to 0.55 and 0.9, the offset to 9 and 2, the floor to 2,
all green. The device-footer width was mutable from 60 to 41 and to 200.
The body-side strip swapped for `String.trim()` was green, and then a body
wrapped in U+00A0 kept its space -- the same Python-vs-Java strip gap the
subject side already pinned. Nine cases added, the five window ones found
by searching for the smallest input each mutant disagrees on rather than
derived by hand. All eight mutants now die.
Two of my own fixes were themselves vacuous, which is the part worth recording.
The NUL check scanned the pattern SOURCE for a raw NUL -- a thing no pattern
can hold, since every class member is written as an escape -- while the two
forms a NUL literal actually takes read as ordinary text to `indexOf`. It is
behavioural now: no pattern may match a lone NUL. And the categories overload
had no test; the one I wrote compared `toString()` on maps of `Question`, which
has none, so it compared identity hashes and was true however the method
behaved. It compares labels now. Both mutants die.
Claims corrected rather than left standing: the filter is not "40 times
cheaper" (it is at parity with the pattern on a paragraph holding no literal,
and worth 1.16x on the corpus -- the 40x was true only while `\b` was a
lookaround); DISCLAIMER has twelve alternations and 26 boundary sites, not
"twenty alternations each carrying two"; the walk is 15.8 ms and the first call
52-57 ms, not the two contradictory figures that were there; `'0'` is safe
because DISCLAIMER holds no digit literal, not because no pattern does -- two
others do. The surrogate figure is now attributed to a one-off sweep that is
not in the tree, and points at the four committed cases that hold the line.
Also: `boundaryView`'s invariant is asserted in code points, not chars -- the
char assertion was false for an astral disagreement and passed only because
every case in the list was in the BMP; an empty character class now throws
something that names the table instead of arriving as `NoClassDefFoundError` on
the second call; a dead `JAVA_WORD` pattern is gone; `isMark` and `isInitial`
use the ASCII fast path the generator was already emitting masks for; the
`@return` says what an extra named `subject` does; and four raw NUL characters
that had ended up in the test source are escapes again.
763 tests pass, identically on JDK 17 and JDK 24. Both sweeps are clean -- 0 of
40,000 and 0 of 20,000 -- every generator gate is clean, email.json regenerates
to identical bytes, and actionlint with shellcheck is clean on the workflow.
What the review found CLEAN is worth recording too, since it is where I
expected the bugs: surrogate handling throughout, integer overflow at
`Integer.MAX_VALUE`, `subList` on an empty body, thread safety of the lazy
holder, mutability of the returned question map, and every pattern but
ATTRIBUTION_TAIL on a 192,000-character hostile line.
The port-fidelity review of the email module, run against the reference line by line. Two findings; the first is a divergence on a narrow input and the second is the kind that is silent. A NEGATIVE max_chars IS NOT A ZERO max_chars. Python slices twice -- once at `max_chars * 4` and once at `max_chars` -- and a negative index drops the LAST |n| code points rather than returning nothing. For a budget of -5 the reference answers "I cannot log in a"; this port answered the empty string. 17 of 171 body-and-budget combinations differed and every one of them was negative. The cut now implements Python's slice for either sign, in code points, and the `len(text) > max_chars * 4` guard goes with it: it is redundant once the slice is faithful, and always true for a negative bound anyway. The test that was here asserted the empty string as correct -- "a non-positive budget returns nothing rather than throwing" -- so it is corrected rather than the fix weakened. That is the third test on this branch found asserting the behaviour it was supposed to be checking. Its replacement reads its expectations off the reference, because the first version I wrote guessed "a mess" for a budget of -3 and was wrong: both slices apply, so a body only survives a budget of -n if it is longer than 5n code points, and "a message" at -3 is empty after the FIRST slice. AN `extra` COULD SHADOW A PARAMETER AND NOTHING SAID SO. In the reference none of these can become a state field: `subject`, `body` and `max_chars` raise a TypeError, and `sender` and `clean` bind the parameter instead. A Java Map has no such protection, so every one of them was being put into the state. `body` is the one that bites -- the state is model input, so the value lands there UNCLEANED, with no quoted history removed, no signature cut and no budget applied -- and a caller translating `email_state(**payload)` got silence where the reference gives an error. All five are refused now, rather than emulating the two that bind: a Java caller has explicit parameters for those, so an entry naming one is ambiguous rather than meaningful, and erroring is the direction that cannot ship the wrong thing to a model. `from` is NOT reserved, because the reference lets it through, and the test pins that it replaces the sender in place rather than being appended. 115 recorded cases match, 768 tests pass identically on JDK 17 and JDK 24, both sweeps are clean -- 0 of 40,000 and 0 of 20,000 -- and both new mutants die. What that review found equivalent is worth recording, since it was looking for exactly the things this port gets wrong elsewhere: the three character classes over all 1,114,112 code points INCLUDING once compiled under the case flags, which widen ranges; `boundaryView`'s soundness over every code point; the sign-off tail scanner against the regex over 145,039 cases; `DEVICE_FOOTER` and `SIGNOFF_ROMANCE` over their full grammars, 91,564 cases; DISCLAIMER against the reference over 20,822 cases with word-class-stressing code points spliced in at every boundary; case folding including U+017F and U+212A; `.` and `$` under UNIX_LINES; and the whole cleaner over 14,720 differential cases. No integer overflow at Integer.MAX_VALUE. Nothing there to fix.
…fore they ship
The three deliverables the PR promised and had not delivered. `laya-java` was
the only one of the three ports with no samples at all; .NET ships three and TS
ships two.
MODELS.md: what `Agent.open` actually takes -- two directories that already
exist -- and the one script that makes both. It documents the `--no-graph` path
first, because most of what this SDK does needs no weights: the tokenizer, the
sequence builder, the collator, detection, the presets, the router's DECISION
and the email cleaner are all exercised by two files that are kilobytes, which
is why the cheap parity cell runs in seconds. It also records that
`typed-decisions` is routable here -- Router knows it, its four aliases resolve
-- but has NO recorded end-to-end fixtures, so the decision is tested and the
forward pass against that checkpoint is not. Every claim in it was checked
against the tree rather than copied from the .NET equivalent: the flags, the
paths, the gitignore, the exporter's arguments, the error text when a graph
directory holds neither layout.
CHANGELOG.md in Keep a Changelog form, and the Known limitations section is
the point of it. It names what is not implemented, the 88-in-200,000 softmax
divergence, the 218-in-1.7M float repr spelling, the seven tokenizer
behaviours no shipped checkpoint declares, and -- under its own heading -- the
seven places Java is not Python, each of which was a defect before it was a
lesson.
THE SAMPLES ARE SINGLE-FILE SOURCE PROGRAMS, NOT A GRADLE MODULE, and that is
a decision rather than laziness. A `samples` subproject would inherit the root
build's maven-publish and get a MavenPublication like every other subproject,
and `publishAllPublicationsToCentralRepository` would push it -- which is
exactly why the empty `laya-java-client` module was taken out of
settings.gradle.kts. A version burned on Central cannot be replaced, and
demonstration code has no business being publishable. They still compile in
CI, with -Xlint:all -Werror, so a sample that stops matching the API fails the
build rather than sitting in the tree misleading a reader.
All three were RUN, not just compiled, and running them found two things a
compile could not:
* Routing.java routed an English state, which would have asked a reader to
export a SECOND 1.3 GB graph to see the same thing. It uses a Portuguese
state now and needs one.
* Benchmark.java warmed up 20 times on a batch of 16 -- about 340 forward
passes, 70 seconds on one line, which reads as a hang. The model-backed
figures are 1 warmup and 2 rounds on a batch of 8 now, and the whole run is
34 seconds. A sample that looks hung is a sample nobody finishes running.
What they print, on this machine: detection 0.085 ms on a 42-character state
and 0.496 ms on 3.8 KB; the email cleaner 0.119 ms; one forward pass 1.32 s and
a batch of 8 at 0.93 s per state, on a 322M model on CPU. Benchmark reports the
COLD first call separately rather than averaging it in, because for this SDK
that figure includes the one-time walk of Unicode -- 66 ms -- and quoting it as
throughput is how a port comes to look slower than it is. It also explains why
the per-state batch figure is only modestly better: batching amortises
tokenization, collation and the session call, and none of those is the cost
here.
Quickstart answers `billing` for both an English and a Hindi state using the
same question objects, which is the point of a question set being plain data,
and reads the answers by pattern-matching the sealed `Answer` interface rather
than by looking up a map -- a choice is not a score, and the compiler can check
that every shape is handled.
Routing shows the three parts that fail in different directions: the decision
and its reason string for four scripts, which needs no model at all; the
load-and-evict lifecycle with `maxLoaded(1)` so the eviction is visible; and a
shortlist of 400 labels down to 5 with a deliberately bad embedder, because the
quality of a shortlist is the quality of the caller's encoder and a bad one
removes the right answer before the model can be asked about it.
768 tests pass identically on JDK 17 and JDK 24, all ten README java blocks
still compile, and actionlint with shellcheck is clean on the workflow.
… 21 it does not
The JDK matrix added two commits ago failed on its first real run, in the cell
it was added to cover. `UnicodeTablesTest` asserted that
`Character.isLetter` DISAGREES with CPython over all 1,114,112 code points --
with a comment saying that if a future JDK ever caught up, the test should fail
so the table could be reconsidered rather than kept out of habit.
A JDK has caught up. JDK 21 ships Unicode 15.0 and this port records CPython
15.0.0, so the two letter sets are identical there and the digests match. JDK 17
(Unicode 13.0) and JDK 24 (16.0) both still disagree, which is why only the 21
cell went red.
Catching up is not a reason to remove the table; it is the reason the table
exists. The agreement is two version numbers lining up by accident and it
un-happens on the next release in either direction, so a port that read the
predicate from the JDK would have been right on 21 and wrong on 17 and 24 --
which is precisely the defect this port shipped in its pre-tokenizer, where the
same jar gave different token ids on different JDKs.
So the test is split along the line that actually matters:
* the two properties the JDK has NO equivalent for are still asserted, because
they disagree on every version rather than on old ones. `str.isspace()`
takes U+00A0, U+0085, U+2007 and U+202F and `Character.isWhitespace`
rejects all four -- a different predicate, not a stale one -- and the
canonical combining CLASS is not exposed by the JDK at all, with Mn/Mc (what
the .NET port uses) differing on 1,528 code points.
* whether `isLetter` and `isUpperCase` agree is now RECORDED and printed, not
required. What is asserted instead is version-independent: that the tables
themselves still match CPython, and that not every JDK predicate has caught
up at once -- which would genuinely be the moment to reconsider.
Verified against a real JDK 21 rather than reasoned about: Temurin 21.0.12.1
reproduces the failure (`isLetter agrees with CPython = true`,
`isUpperCase = true`) and passes with the fix.
JDK 17 IS NOW IN THE MATRIX TOO. The build job already tests on 17, but without
`-PtestJavaVersion`, so `TestJvmVersionTest` is disabled there and "17 is
tested" was true but unasserted. The three cells are also three different
Unicode versions -- 13.0, 15.0, 16.0 -- which is the axis this port's whole
character-handling strategy is about, and a two-version matrix would not have
found this.
One thing the failure hid: the old assertion was the FIRST in its test, so when
it failed on 21 nothing else in that cell was ever checked. The model-free suite
now runs clean on all three JDKs, 626 tests each, identically. 769 with the
checkpoints and the fused graph attached.
…can move The drift gate failed on CI and reproduced exactly here, with identical numpy, torch, transformers and tokenizers pins -- so not a toolchain difference. The cosines in `shortlist.json` were rounded to twelve decimals, and twelve is fine until a value lands within the accumulation error of a rounding boundary. Measured rather than assumed: the worst difference between numpy's `dot` and a sequential sum over 4,000 random unit-vector pairs is 2.22e-16, about one ULP near 1, and these vectors are only eight-dimensional. With 194 floats in the file the chance that at least one straddles a twelve-decimal boundary is about 8.3%. At nine decimals it is 0.0085%. The gate was a coin flip rather than a contract, and the comment on `_round_score` said "a precision BLAS blocking cannot move" -- which was simply false and is corrected rather than softened. Nine is three orders of magnitude tighter than any porting error this repository has seen -- a wrong normalisation, a missing clamp or an inverted tie order moves a score in the third decimal, not the tenth -- and seven orders looser than the measured spread. What is still compared EXACTLY is the label order, which is the only thing a consumer observes and the thing a ranking bug moves. `scorePrecisionIsHonest` did its job: it pins the fixture's precision and the comparison tolerance to each other so neither can drift alone, and it failed the moment the fixture changed. Both move together here, along with four comments that asserted twelve decimals in prose -- including the two on the zero-norm and clamp tests, which explain why those are asserted EXACTLY precisely because the rounded comparison cannot see 1.4e-200 or 1.0000000000000002. AND THE GATE NOW SHOWS WHAT DRIFTED. `--check` printed "shortlist.json is stale" and nothing else: not which value, not by how much, not whether it was one float or the whole file. Diagnosing a cross-platform float difference from that took three CI round trips, two of them spent on the wrong hypothesis. On failure the step now regenerates and prints the diff, which costs nothing on the passing path. 626 model-free tests pass on JDK 17, 21 and 24 identically, the three samples still compile with -Werror, and every fixture family matches.
The previous commit was the wrong fix. It lowered the recorded precision from
twelve decimals to nine, which is a real improvement for the `cases` section --
but the values that were actually drifting are in `predicts`, and they had no
rounding whatsoever.
`predict_shortlist`'s result was handed through verbatim:
"shortlist": result["shortlist"],
so the reference's `Ranking` objects kept their raw float64 scores -- sixteen
and seventeen significant digits -- while the same numbers in `cases` went
through `_round_score`. The diff the build lane now prints says it outright:
- 0.5259653204777385 + 0.5259653204777384
- 0.45371261735814744 + 0.4537126173581474
- 0.12186924644473365 + 0.12186924644473371
Sixteen decimals, not nine. Changing the rounding precision could never have
fixed those, because nothing was rounding them. Three values differed in their
last bit between x64 and arm64, which is exactly what a raw float64 cosine does.
THE DEFECT WAS A MISSED CALL SITE, NOT A PRECISION, so the guard is on the
OUTPUT rather than on the code. `_assert_scores_are_rounded` walks the whole
payload and refuses to emit a shortlist fixture holding any float under a
"scores" key that is finer than the recorded precision, at any depth. Reverting
just the one-line fix makes it name all eleven offending values and their paths
instead of writing the file. A grep for `_round_score` found two call sites and
both were correct; what was wrong was a third place that needed one and had
none, and only a check on what was written could see that.
Also swept the other six byte-gated families for the same shape. Two hold
full-precision floats and both are safe: `lang_detect`'s 68 are hand-written
rounding-boundary literals like 0.49999999999999994 that the round-to-int cases
exist to test, and `shortlist`'s remaining 26 are `embed_probe`, which comes
from an FNV-1a hash and integer division -- bit-exact on any platform.
`python_json`'s two are deliberate: that family tests float repr. Nothing else
computes a float through BLAS.
The nine decimals stay. They were not the bug here, but twelve was still a
coin flip for the `cases` section -- 8.3% chance that one of 194 floats straddles
a boundary -- and the measurement behind that is recorded on the constant.
626 model-free tests pass on JDK 17, 21 and 24 identically, every fixture family
matches, and regenerating twice gives identical bytes.
… branch
Brings the three remaining `laya-java` items onto `feat/laya-java-routing`,
each already carrying its own adversarial review and the fixes that review
demanded:
* `typed-decisions` end to end, plus the identity check that now gates the
forward pass instead of running ninth of thirteen.
* `Decisions.decide` / `decideBatch` against a JSON schema, plus the refusal
of a non-finite score that used to narrow silently into the declared range.
* the prediction hooks, plus the twelve defects an independent sweep found in
them -- six surviving mutants, three documented contracts with no test, and
a deadline that bounded the wait rather than the request.
Three things could only be settled here, on the merge, because no branch could
see them alone.
`build.gradle.kts` keeps BOTH halves of its environment block. One review found
`LAYA_TYPED_ONNX_GRAPH` missing from the pass-through list; a different review
found that `environment(...)` is not a Gradle task input, so toggling a lane
left `test` UP-TO-DATE and handed back the previous lane's results. Neither fix
supersedes the other and taking one side would have dropped a real one.
`scripts/gen_fixtures.py` is merged structurally, not textually. Git aligned the
hunks on `}` and `try:` and cut them mid-function, which does not parse. The two
sides add 27 and 17 top-level blocks with no name collision, and both edit only
`FAMILIES`; the result is their union, 18 families, asserted to parse with no
duplicate definitions.
The test-count floor is re-derived rather than reconciled. Each branch raised it
against its own base -- 779, 686/687, 813/814 -- and every one of those leaves
over a hundred tests of slack against the merged 917, so each would have been
decorative. Measured here: 934 register and 917 run with no checkpoints, 918 in
the `jdk-matrix` lane because `-PtestJavaVersion` is what enables
`TestJvmVersionTest`, and 1,328 of 1,334 with both checkpoints present. The
floors are EXACT, with no margin, which replaces a margin of 4: three classes in
this lane run exactly one test, and no positive margin can bind a one-test
class. Proved by deleting `WindowPlanTest` -- the suite stays green at 916 run,
the exact floor exits 1, and 813, 779 and 686 all exit 0.
`README.md` and the CHANGELOG state what is true after the merge rather than
what each branch knew: hooks, `decide`, `predictLong` and `ConfidenceGate` all
exist, so only the HTTP client module and Android remain unimplemented.
|
@NandhaKishorM PR pending your review. |
`java/unreleased-lock`, high, in the blocking `security-extended` suite, on `Hooks.dispatch` -- the one place where a stuck lock wedges every hooked call rather than one. The code was not the naive form. `lock()` and `unlock()` were already paired across a `try`/`finally`. The defect is that `policy.lock()` was CALLED TWICE, once for each half. `Policy` is a record, so both calls return the same monitor today and the path is not reachable; nothing in the type requires that, and locking one instance while unlocking another leaves the first held with no owner able to release it. Read once into a local now. Neither semgrep nor an independent 14-mutant sweep over this file saw it. It is not a property of any line -- it is a property of which paths reach the unlock, which is the whole reason CodeQL is in the lane. This is the second unreleased lock it has found in this module that the pattern scanner called clean. Measured: 14 findings before, 12 after; `unreleased-lock` goes to 0. The suite is unchanged at 934 registered, 917 run, 0 failed. `Hooks.seconds` also takes an explicit guard for a missing exponent marker. It is unreachable from a `Duration`, whose `toNanos()` is a long so the value is always finite -- but the method is public so a caller's own failure sink can emit the same line, and `%.5e` of a non-finite double renders "Infinity", which holds no 'e', so `substring(0)` would hand the whole word to `Integer.parseInt`. Stated plainly: this does NOT clear the `uncaught-number-format-exception` finding, which merely moves to the new line number -- CodeQL cannot prove the substring is numeric whatever guards it. It is now the same accepted class as the two pre-existing ones in `PythonJson`. `%g` parity with CPython re-verified in both runtimes afterwards, byte-identical on ten values including 1e-07. The four `java/unused-parameter` hits on `Hook`'s default no-op methods are left alone: an unused `ctx` is what a no-op default IS, and `Router.java:111` is the same finding already accepted at base.
Comment-only; `git diff` shows zero non-comment lines changed. Both `check_test_results` steps already carried the right floors, 917 and 918, measured on the merged tree. The prose above them did not. The build lane's block still read "703 register here, 686 run ... so it executes 687" -- the typed-decisions branch's figures, sitting four lines above a floor of 917. The jdk-matrix lane's block still read "814, not the build lane's 813 ... 818 run against a shared floor of 813" -- the hooks branch's figures, above a floor of 918. Each was correct on the branch it was written on and wrong the moment four branches merged, which is the same failure this PR documents in the code it reviews: a justification that reads as measured while the number it justifies has moved underneath it. Mine, this time. Both now state the merged measurement, and the jdk-matrix one says why the two lanes differ by exactly one rather than repeating a margin argument that no longer applies. Also records what CI has since settled. That comment said the exact floor was prepared without a JDK 21 to test on, so whether 21 registers the same count as 17 and 24 was an argument -- no JDK-version-gated assumptions exist in the suite, and 17 and 24 agreed -- rather than a measurement. All three lanes have now run green at these floors, so it is a measurement, and the comment no longer claims otherwise. Kept as deliberate history, not stale: the "(779, 686, 813)" list of what the four branches each set against their own base, and the 794/793 account of the shared floor that let an emptied fixture section pass invisibly.
…nload pressure
`.NET / parity (multilingual)` has failed twice on this branch while passing on a branch
that adds no parity cell, so it is not the flake I first called it, and the mechanism
implicates this change.
From the failing job's own export step:
Warning: You are sending unauthenticated requests to the HF Hub.
Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Fetching 5 files: 100%|##########| 5/5 [00:04<00:00, 1.03it/s]
Five files including a 34 MB `tokenizer.json`, in four seconds, unauthenticated. `hf-xet`
returned a TRUNCATED file and did not raise: the download step reported success and the
failure surfaced ninety seconds later as `EOF while parsing a value at line 1 column 0` from
inside a tokenizer constructor -- "line 1 column 0" being an empty file. 19 of 302 tests
failed, every one a multilingual tokenizer case, while english, router and typed-decisions
all passed in the same run. A code defect does not empty one checkpoint's file and leave the
others intact.
This matrix gained a third weight-downloading cell in this change, and `dotnet.yml`'s parity
matrix runs four of its own against the same repo in the same workflow run, with
`max-parallel` unset on both. Peak was seven concurrent unauthenticated fetches of one repo.
`max-parallel: 1` here drops that by two and costs wall-clock on a lane that is already
advisory.
Stated plainly: this is a MITIGATION of pressure this change added, not a fix for the
defect. The defect is that nothing verifies a download before using it, so a truncated fetch
becomes a confusing parse error three minutes downstream instead of an immediate, obvious
failure. The fixes for that are an `HF_TOKEN` for the higher rate limit, or a size or digest
check at the fetch -- both in `laya-dotnet/tools/` and `dotnet.yml`, which this PR does not
touch, so neither is bundled here rather than widening it into another module's lane.
Earlier I attributed this to a transient download and said this PR could not be the cause.
The twice-on-this-branch, pass-elsewhere pattern does not support that, and the comment in
the workflow now records what actually happens instead.
`.NET / parity (multilingual)` failed twice on this branch and passed on a sibling. From the failing job's own log: the hub rate-limited an UNAUTHENTICATED request and `hf-xet` wrote a 0-byte 34 MB `tokenizer.json` without raising. `snapshot_download` reported success; the failure surfaced ninety seconds later inside a .NET tokenizer constructor as `EOF while parsing a value at line 1 column 0`, a message that names no download. 19 of 302 tests failed, every one a multilingual tokenizer case, while english, router and typed-decisions passed in the same run. Three parts. `verify_snapshot(directory, label, required)` at the three download sites CI reaches -- `laya-dotnet/tools/regen_golden.py`, `laya-dotnet/tools/export_onnx.py` and `laya-java/scripts/prepare_checkpoint.py` -- refusing a snapshot whose requested files never arrived, or that holds a 0-byte file, unparseable JSON, or a `.safetensors` too small for a header. `required` is the load-bearing half: walking what is present cannot see a file that never came, and an empty directory passes a walk. `max-parallel: 1` on both parity matrices. Every cell needing weights fetches the same repo, and four in `dotnet.yml` plus three here in one run is what provoked the throttle. Peak concurrent fetches drops 7 to 2. Costs 7m38 of serial wall-clock here and 19m12 there, on lanes that are advisory. `HF_TOKEN` on the three fetching steps -- scoped to those steps, not workflow-level, because a workflow `env` reaches `pip`, `gradle`, `semgrep` and every third-party action in both files. The review of the first version of this found two defects that would have made it worse than the bug, and both are worth recording. `verify_snapshot(..., name)` was written where the enclosing parameter is `name_or_path`, so every Hugging Face download would have raised `NameError` and all four .NET parity cells would have failed unconditionally -- and because this change alters the files the export caches hash, the cache miss guaranteed the path ran. `ruff --select=F82` finds it in under a second, so `ci.yml` now runs that group over the three SDK script directories as well as `laya/`. Split into a second invocation: those directories carry pre-existing F401/F811 import debt that is not worth churning here, while F82 is a bug class rather than style. Worse, the guard REJECTED a healthy checkpoint. With `local_dir=`, `huggingface_hub` keeps bookkeeping under `<dir>/.cache/huggingface/download/` whose `.lock` files are legitimately 0 bytes; english has no subfolder, so the verified directory was the `local_dir` itself and `jvm parity (tokenizer)` would have failed every run. `.cache/` is now skipped, and the real english checkpoint is verified to pass. A fail-closed check that refuses correct input is the one failure mode it must not have. Also from review: an unset `HF_TOKEN` secret expands to `""`, and the explicit `token=` call site passed it through, producing `authorization: Bearer ` and `httpx.LocalProtocolError: Illegal header value`. That is a client-side refusal, not a 401, so no server probe finds it, and every fork PR would hit it. Normalised to None at the call site. The `laya-ts` copy of the guard is dropped: `regen_golden.py` invokes that exporter with `--model-dir`, which short-circuits its download, so it is unreachable from CI and the file ships to npm. Verified: the guard accepts the real english checkpoint including its 0-byte `.lock` files, and refuses a 0-byte `tokenizer.json`, a truncated one, a missing required file and an empty directory, identically at all three sites. Both ruff invocations exit 0 on this tree and exit 1 on the reinstated NameError. `check_workflow_folding` clean, actionlint clean, all three workflows parse. Model-free suite unchanged at 934 registered, 917 run, 0 failed. Stated plainly, since the guard is not a complete one: a `.safetensors` truncated above 1024 bytes still passes, and the real one is ~2.3 GB. Closing that needs the hub's `siblings` sizes or a `safetensors` header length check, which is a larger change than this failure warrants.
The previous commit said a `.safetensors` truncated above 1024 bytes "still passes" and that
closing it needed the hub's `siblings` sizes or a header-length check, "a larger change than
this failure warrants". The first half is true of `verify_snapshot`; the second half was not
measured, and it is wrong.
`safetensors` already validates that the tensor byte ranges cover the file. Measured: the
842 MB english `model.safetensors` truncated to 500 KB raises
SafetensorError: Error while deserializing header: incomplete metadata,
file not fully covered
in the exporter, seconds after the fetch, naming the actual defect. Nothing needs closing and
no follow-up is owed.
That is the whole distinction this guard exists for. The 0-byte `tokenizer.json` was NOT
self-diagnosing -- `EOF while parsing a value at line 1 column 0`, from a .NET constructor, in
a test, ninety seconds later, naming neither a file nor a download. A loader that already says
"file not fully covered" needs no help; one that says "EOF at line 1 column 0" does.
So the 1024-byte test stays a floor for a file too short to hold the 8-byte length prefix and
a header at all -- where the loader's own error would be about a malformed header rather than a
truncated download -- and the comment now says that instead of leaving a reader to infer a gap.
Comment-only; zero non-comment lines changed.
feat(java): language routing, presets and the embedding shortlist
#1013 fixed `_named_prose_language` naming a line of all-caps English acronyms as foreign prose, and added `_SHOUTED_MIN_STOPWORD` / `_SHOUTED_MIN_WORD`. The Java port asserts against fixtures recorded from the Python package, so #938's copies went stale the moment both landed. Regenerated with `laya-java/scripts/gen_fixtures.py`. Recorded on CPython 3.12 (Unicode 15.0.0), the version java.yml pins. Regenerating on 3.11 instead rewrites every Unicode table digest and `unicode_version` to 14.0.0, which is the drift this fixture set exists to catch rather than absorb. The behavioural delta is one case: a segment that read `fr` from `MON LA EST COM DES` now reads `pt` from the Portuguese sentence actually in the state. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
The routing, presets, shortlist and email half of this is merged in 0.3.29, and the folded-block defect is fixed, so thank you for both. What landedConfirmed on run: >
python3 scripts/check_test_results.py laya-java/build/test-results/test
--min-tests 680 --allow-abortedwith the comment moved out of the folded block, so the floor binds. I re-parsed every workflow in the branch for the same shape and found none, and The wrapper validation survived the rebase and you extended it from one job to all four, which is what I would have asked for. What did not land, and whyI merged revision So hooks, Please rebase onto current
On the sizeThis grew from 35k lines to 71k to roughly 108k across two reviews, and each time the new half was good. I am not going to ask you to split what is already written, but for the next piece after this one: the reason I merged a revision you had moved past is that the branch kept growing faster than I could review it, and the version I had reviewed was green. A branch that lands in pieces gets reviewed; a branch that keeps absorbing its own follow-ups gets reviewed at whatever revision the maintainer happens to fetch. Landing hooks as its own PR on top of current One note I could not check: there is no JDK on this machine, so I did not run the Java suite. The fixtures regenerate cleanly and Your |
|
The routing, presets, shortlist and email half of this is merged in 0.3.29, and the folded-block defect is fixed, so thank you for both. What landedConfirmed on run: >
python3 scripts/check_test_results.py laya-java/build/test-results/test
--min-tests 680 --allow-abortedwith the comment moved out of the folded block, so the floor binds. I re-parsed every workflow in the branch for the same shape and found none, and The wrapper validation survived the rebase and you extended it from one job to all four, which is what I would have asked for. What did not land, and whyI merged revision So hooks, Please rebase onto current
On the sizeThis grew from 35k lines to 71k to roughly 108k across two reviews, and each time the new half was good. I am not going to ask you to split what is already written, but for the next piece after this one: the reason I merged a revision you had moved past is that the branch kept growing faster than I could review it, and the version I had reviewed was green. A branch that lands in pieces gets reviewed; a branch that keeps absorbing its own follow-ups gets reviewed at whatever revision the maintainer happens to fetch. Landing hooks as its own PR on top of current One note I could not check: there is no JDK on this machine, so I did not run the Java suite. The fixtures regenerate cleanly and Your |
…arrying NandhaKishorM#938 was merged at 561c496 -- the `Tokenizer.decode` commit -- so everything after it is still open: `hooks/`, `Decisions.java` and `fixtures/typed_decisions.json` are absent upstream. upstream/main has since moved 158 commits, which is what made this conflict. Of the 51 commits on this branch, 27 have a same-subject twin upstream: they landed with that merge under different SHAs. 24 are genuinely new. 13 files conflicted. Resolved per file rather than by preference. For ten of them the test is mechanical -- upstream has not touched the file since my work merged, so this side is upstream's content plus the new work, and `--ours` loses nothing. Three needed a decision: `decode/Decoder.java` takes UPSTREAM. My own comment in that file deferred the question: an empty action block padded to two and published {0.5, 0.5}, and the comment said changing it "belongs in its own change against that test rather than buried in this one; raised with the maintainer separately". The maintainer decided -- it throws now -- and taking this side would have reverted their decision on a question I handed them. `fixtures/lang_detect.json` takes UPSTREAM. It was re-recorded by 7a2f1bb after the Python lang fix landed as NandhaKishorM#1013, so upstream's copy pins the fixed behaviour and this branch's copy pins the bug. `.github/workflows/java.yml` takes OURS, verified rather than assumed. Upstream added one `gradle/actions/wrapper-validation` step in dc72133; this side has it in all four jobs that run `./gradlew`, each before that job's first `./gradlew`. Parsed the YAML to confirm, because dropping a security step is the worst thing a merge can do quietly. Two tests then failed, and BOTH fail on pristine upstream/main at e08843b with nothing from this branch in the tree. They are inherited, not caused here. `DecoderTest` is fixed here, because its assertion had stopped matching the code it guards. `Decoder.actionProbabilities` throws on a zero-width block while the test asserted {0.5f, 0.5f}. The implementation is the half that is right: 0.5 is a number no model produced, published into the field callers gate escalation on, and the reference raises there too -- `laya/onnx_agent.py` indexes `act[offset + r, 0]`, so zero columns cannot return. The test now asserts the refusal and its message, and a mutant that disables the refusal fails it. `LanguageDetectionParityTest` is NOT fixed in this commit, and the reason is worth stating. `english-note-with-acronym-line-and-foreign-line` expects `pt` and the port answers `fr`: the fixture was re-recorded from the fixed `laya/lang.py`, but `lang/LanguageDetection.java` was never updated to match, so the two have diverged in exactly the way the parity gate exists to prevent -- and in exactly the way NandhaKishorM#1013's own PR body warned they would if the fixture moved without the port. Porting that fix is a real change against 116 recorded cases and belongs in its own commit, not inside a merge. Verified: `gradle build -x test` clean under -Werror with javadoc; 924 tests complete; the only failure left is the LanguageDetectionParityTest case above.
…pected The Python side merged the caps bars (NandhaKishorM#1013 and its follow-up); the Java port never got them, so LanguageDetectionParityTest has been failing on main for the one recorded case where an all-caps line of English abbreviations outvoted a genuinely Portuguese line. MON, LA, EST, COM and DES are hockey teams, states, time zones and radio bands that collide with French stopwords. Before this, the bare-acronym line was named fr and won the whole-state verdict. Ported from laya/lang.py: - SHOUTED_MIN_STOPWORD and SHOUTED_MIN_WORD, with the reference's sweep recorded on each. - shouted(): a cased segment written entirely in capitals. The uppercase half is not redundant -- a caseless script has no lowercase either, and would otherwise be held to a bar its tokens cannot clear. Both halves go through the recorded case tables, not the JDK's isUpperCase/isLowerCase, which disagree with Python on 240 code points. - shoutedEvidence(): a word of SHOUTED_MIN_WORD letters, and then either non-English diacritics or a matched stopword of SHOUTED_MIN_STOPWORD letters. - nonLatinCarries(), named rather than left inline, because the caps bars ask the same question to decide what to keep out. - the bar applied in namedProseLanguage, and again in analyseText so the whole-state verdict obeys the module's own rule that an acronym is not a word. Vetoing there leaves the text undecided, so looksNonEnglish still decides it. - only acronym-shaped runs are blanked, via upperLetterRuns(). Blanking every all-caps run deletes emphasis capitals, which are ordinary in prose and usually sit on the words that carry the sentence. blankUpperRuns() itself is unchanged: it is a recorded helper the fixture pins, so analyseText filters before calling it. - leafNonEnglish(): a veto is not a verdict of English. A field whose named language the prose scan disbelieves now falls through to the diacritic branch instead of being dropped. The fixture is not regenerated. lang_detect.json was re-recorded upstream from the fixed reference and is the target here. ShoutedAcronymBarTest asserts the behaviour directly, because the fixture cannot: of the seven mutants used to check it, only one is caught by the parity suite at all. Deleting the analyseText block, blanking every all-caps run, reverting the leaf fall-through, dropping the non-Latin exemption and loosening either bar by one letter all leave LanguageDetectionParityTest green. Suite: 934 tests 1 failed -> 947 tests 0 failed. Parity 116/116.
…n produce
`gen_fixtures.py --check` went stale on `confidence_gate.json` after the merge, and following
that to its cause found a behaviour divergence rather than a recorded-message drift.
Upstream `cb85656` changed `laya/confidence.py` from "a key must be a string" to "a key must
name a bucket an answer can produce". This port only ever checked for null, so it accepted keys
the reference refuses:
'choice:3-5' 'default' 'choice:99' 'nonsense'
reference accept accept REFUSE REFUSE
this port accept accept accept accept
now accept accept REFUSE REFUSE
That is worse than a wrong message. A threshold filed under `choice:99` is never applied to
anything, because no answer resolves to that bucket -- the caller believes they gated a decision
and nothing is gated. Accepting the key the reference refuses turns a loud error into exactly
that silence, which is what this gate exists to prevent.
The allowed set is derived from the same spelling `optionBucket` emits -- three question types
crossed with the four size bands, plus `default` -- rather than written out, so the two cannot
drift. The message is matched character for character including `%r` quoting and `None` for a
null key, verified against the merged Python on the same four inputs.
`fixtures/confidence_gate.json` is regenerated on **CPython 3.12**, which `java.yml` pins. One
leaf moved, the refusal message. 3.11 would have rewritten `unicode_version` to 14.0.0 and every
Unicode digest with it, which the maintainer lost a commit to; the fixture records 15.0.0 and
3.12.2's `unicodedata` is 15.0.0. All 18 families now match.
The fixture could not have caught this. Its recorded refusal uses the key `1`, an int, and this
port's signature is `Map<String, ? extends Number>`, so that case is unrepresentable here and the
comparison skips it. A bad STRING key is representable and nothing recorded reaches that path, so
`ConfidenceGateTest` sat at 26/26 green over the gap. `ConfidenceGateBucketKeyTest` covers it,
including a case that feeds every bucket `optionBucket` can emit back in as a key so the allowed
set and the bucketing function cannot diverge. A mutant that drops the check to the old null-only
test kills it: 3 tests, 1 failure, compiled clean.
Floors re-derived, which the maintainer asked for explicitly: 950 register, 933 run, so 933 and
934. Proven to bind by deleting a one-test class -- 929 run, the floor exits 1 -- and every piece
of prose above each floor moved with the number, because a justification above a changed figure
is the defect this branch has already shipped twice.
…atters `max-parallel: 1` on both parity matrices cost more than it bought. Measured from this PR's own cells, serialising takes the parity critical path from 10:56 to 30:08 -- java 2:24 + 5:14 + 8:05 against 8:05 parallel, dotnet 7:52 + 7:28 + 10:56 + 3:52 against 10:56 -- which is nineteen minutes added to every run to reduce the chance of a fetch being throttled. Two keeps the peak well below the seven concurrent unauthenticated fetches of one repo that provoked the throttle, at roughly half the wall-clock. The reason this is safe to relax is that serialising was never the fix. `regen_golden.py`, `laya-dotnet/tools/export_onnx.py` and `laya-java/scripts/prepare_checkpoint.py` now refuse an incomplete snapshot AT THE FETCH -- a file that never arrived, a 0-byte one, unparseable JSON -- so a throttled fetch fails immediately with the file named, instead of surfacing ninety seconds later as `EOF while parsing a value at line 1 column 0` from inside a tokenizer constructor. `max-parallel` only changes how often that path is taken; the guard is what makes taking it survivable. The earlier commit said as much and then chose the expensive half anyway. Comment-only apart from the two values: `git diff` shows four non-comment lines, the old and new setting in each workflow. Note for whoever watches the next run: this PR also changes all three files that are hashed into the export cache keys, so the first run after it is cold in every parity cell regardless of this setting. That cost is one-time and is not what this commit addresses.
Fixed and rebased @NandhaKishorM |
A choice's options are positional: option N renders Nth and logit N is read back
as the Nth label. So the criteria map's iteration order decides both what is
asked and how the answer is labelled -- and `choice` accepted any `Map`, copying
it into a `LinkedHashMap` that faithfully preserved whatever order the caller's
map happened to have.
Measured on JDK 17, options written `refund, escalate, ignore`:
LinkedHashMap refund, escalate, ignore
HashMap ignore, escalate, refund
Map.of varies per JVM run
`HashMap` is exactly reversed, so `probabilities().get("refund")` reported the
logit that belonged to `ignore`. That much is at least reproducible, because
`String.hashCode` is specified. `Map.of` is worse: it salts its table per JVM, so
three runs of one program gave `[ignore, escalate, refund]`,
`[escalate, ignore, refund]` and `[refund, escalate, ignore]` -- the same question
answering differently after a restart, with nothing in the output to say so.
`Map.copyOf` returns that same implementation, so copying a `LinkedHashMap`
through it destroys the order it was built to carry.
`choice` now refuses such a map and names the fix. A denylist rather than an
allowlist, because a false refusal breaks working code: Guava's `ImmutableMap` and
`Collections.unmodifiableMap(aLinkedHashMap)` both iterate in a defined order this
check cannot see. Checked by `getClass()` rather than `instanceof`, because
`LinkedHashMap` IS a `HashMap` and an `instanceof` test would reject the one map
the method most wants to accept. A single-option map is always accepted: one
option cannot be out of order.
`Question.choiceOf(instructions, Map.entry(label, description), ...)` is the
counterpart for callers who would otherwise reach for `Map.of`: no intermediate
collection, so there is no order to lose. It refuses a duplicate label, which a
map would have resolved by keeping only the last -- asking a question with fewer
options than were written. The check is `containsKey` rather than the return of
`put`, because a legitimate criterion may be null and then `put` returns null for
a duplicate too.
Pinned by ten tests, and both halves are pinned by a mutant: neutering the guard
fails four of them, removing the duplicate check fails one.
Floors moved with the count, which is exact by design and has no margin:
java.yml model-free 933 -> 943
java.yml jdk-matrix 934 -> 944 (+1 is --require-passed TestJvmVersionTest)
release-java.yml 680 -> 943
The release floor was stale enough to be no gate at all: against a measured 943 a
suite could have lost 263 tests and still passed it. Its comment said "685 of the
699 registered tests run without a checkpoint", which is also now the measured
943 of 960.
build -Xlint:all -Werror clean
model-free suite 960 registered, 943 run, 0 failed
real-weights e2e 1367 tests, 0 failed, 0 errored
check_workflow_folding 10 workflows clean
The e2e ran both local checkpoints and the exported graph: TokenDecoderTest 195,
LanguageDetectionParityTest 116, TokenizerParityTest 86, WindowPlanTest 66,
SequenceParityTest 52, PredictLongTest 49, DecoderParityTest 40,
PredictParityTest 9. `TypedDecisionsParityTest` ran 4 and skipped 3: the
typed-decisions checkpoint is not on this machine.
… the new default
`3cf26cb` ("default to the multilingual checkpoint for undecided text") changed
`laya/router.py` without regenerating `laya-java/fixtures/router.json`. GitHub
tests a PR merged with main, so the stale fixture failed here even though nothing
on this branch caused it:
gen_fixtures: router.json is stale
##[error]Process completed with exit code 1
`router.json` is stale on `upstream/main` by itself, so this failed on every open
PR. Timing: b7cc4da passed every job at 11:16, 3cf26cb landed at 14:14, 253b5c2
failed three jobs at 19:20.
Regenerating it exposed the real divergence the fixture exists to catch: both
ports still defaulted to english.
laya-java/fixtures/router.json regenerated, 24 lines
Router.java Builder.defaultCheckpoint ENGLISH -> MULTILINGUAL
LayaRouter.cs Default English -> Multilingual
Identified English still routes to english on both; only the undecided default
moved, which is all 3cf26cb changed. Two Java tests and two .NET route cases
asserted the old default and are updated. `RouterLifecycleTest` was asserting
"plain english text here now" -> ENGLISH; `analyse` gives that string
language=null, undecided=true, so it was never identified English and only landed
on english through the default.
Also fixes three findings from an adversarial review of 253b5c2:
HIGH -- `orderIsUnspecified` used `getClass()` equality, so a subclass slipped
through: `class Mine extends HashMap {}` was accepted and published option 0's
probability against option 2's label. `instanceof` now, which is safe because the
ordered types return before that point -- and the comment justifying `getClass()`
was simply wrong about why.
HIGH -- `Shortlist.rank` copied the caller's map into a `LinkedHashMap` before
calling `Question.choice`, so the guard could never fire there. Order decides
which labels survive the cut in `Shortlist`, not just how they are numbered, so a
`HashMap` dropped a different option. Validated before the copy now.
HIGH -- the README claimed `choice` "refuses a map whose contract leaves the order
unspecified". It does not: `Collections.unmodifiableMap`/`synchronizedMap`/
`checkedMap` are the same wrapper class whichever map they wrap, so a wrapped
`HashMap` is accepted. Reaching the wrapped map needs `--add-opens` into
`java.base`. The README and javadoc now say what is and is not caught.
Three tests added for the two holes, and five stale measured counts in
java.yml's comments corrected.
build -Xlint:all -Werror + javadoc clean
model-free suite 963 registered, 946 run, 0 failed
gen_fixtures --check 18/18 match
check_workflow_folding 10 workflows clean
Floors: java.yml 946 and 947, release-java.yml 946.
Not verified locally: the .NET change. `dotnet` is not installed on this machine,
so `LayaRouter.cs` and the two route cases are reasoned from the same Python
reference the Java fix was verified against, and CI is the first run of them.
…point
`LayaRouter.Default` moved to multilingual with the reference, but the committed
routing goldens still recorded english on the default path: 8 entries reading
`using default (english)`. CI regenerates these from Python on every run and the
drift check against the committed copy is only informational, so the fresh
goldens said multilingual while the committed ones said english -- which is the
mechanism behind the two failing .NET jobs.
Regenerated with `tools/regen_golden.py --checkpoint routing`: 7 `model` and 7
`reason` fields, nothing else. One `using default (english)` remains, for the case
that sets `default="english"` explicitly. A second generator run is a no-op, so
the output is stable.
Verified in a container, since this machine has no dotnet
(mcr.microsoft.com/dotnet/sdk:10.0, arm64, SDK 10.0.401):
dotnet build Laya.slnx -c Release 0 Warning(s), 0 Error(s)
Laya.Tests, CI's five exclusions Total: 709, Failed: 0, Errors: 0, Skipped: 0
Note for anyone reproducing it: `dotnet test --solution Laya.slnx` reports "No
test projects were found" in that image -- `IsTestProject` evaluates empty. The
project is xunit.v3 with `OutputType=Exe`, so the test assembly is its own runner;
running `bin/Release/net10.0/Laya.Tests` with `-class-` per excluded class is the
equivalent and works.
Follow-up to #927, which merged while still marked a draft and so landed
laya-javaas the runtime only. This PR carries the rest of the SDK, so thatlaya-javaends up complete in one piece the waylaya-dotnetdid in #671.Updated since first opened. Four further items have landed here —
predictLong, the abstention gate, structureddecide, and prediction hooks — plus end-to-end coverage of thetyped-decisionscheckpoint, which this PR previously declared routable but unrun. Each of those four was then put through an independent adversarial review, and every one of the four reviews found real defects; the fixes are separate commits so the two are reviewable against each other. There is a section on that below, because the pattern is more useful than the individual bugs.It opens with the routing layer: the part that decides which checkpoint a request goes to, and therefore the part that matters most for anyone whose traffic is not all English.
Routing is not a tuning question. The English checkpoint does not degrade gently off English, it collapses — 0.100 on 20-option MASSIVE Hindi intent against 0.050 for random guessing, while reporting high confidence (ECE 0.855). The multilingual checkpoint gains 21 points on non-English XNLI and gives up about 6 on English suites. A wrong route is not a slightly worse answer; it is a confident wrong one.
Three of these commits fix defects that are on
mainnowLeading with this, because it is buried under "What the review of #927 found" below and it is the part with a deadline.
maintodayrelease-java.yml*-SNAPSHOTis caught--min-tests 80, andcheck_test_results.pycompares JUnit'stestsattribute — which counts skipped tests\p{L}underUNICODE_CHARACTER_CLASS, so the class follows the JDK's Unicode version..., null, false, stateIds)— a literalfalsestate instanceof ListThe first two compose into something that cannot be undone: on a
workflow_dispatchfrom a branchGITHUB_REF_NAMEis the branch, so the published version becomesmain, and Maven Central is append-only. The floor that was supposed to stand in the way is satisfied by a run in which every test was skipped.The other two are wrong answers rather than wrong releases. The same jar gives different token ids on different JDKs — Corretto 17 carries Unicode 13.0 and Corretto 24 carries 16.0, and they disagree on 9,917 code points, including the ordinary frame
"The X's value". And an oversized conversation was cut from the wrong end, so the model saw the opening of the conversation and answered about that while the current turn was discarded;usagewas byte-identical either way, so nothing a caller can read revealed it.If the rest of this is too large to take at once, those four are
be6b398,0e1f1b1and01a13b7, and each is green on its own.What is in it
lang/LanguageDetection.javalaya.langin full: script counting and profiling, the non-Latin word scan, the Latin-language guess with all four cascade branches, the acronym/code/compound exclusions, the per-line segment scan and the per-leaf scanPresets.javastateFieldRouter.javareasonstrings, and the load/lease/attach/preload/unload/predict lifecycleShortlist.javalaya.shortlist: the cosine ranking, the stable tie order, the passthrough, and the LRU embedding cachePredictor.javaAgentandRouter, so shortlisting works against eitherLayaEmail.javalaya.email: the cleaner and the state builder, with the English, Portuguese, Spanish and French marker setsjson/PythonJsonreprand%.0f, which those reason strings are built fromAgent.predictLongLongPredictionConfidenceGate.javaUNEVALUATED, which a boolean cannot expressDecisions.javalaya.structured'sdecide/decideBatchagainst a JSON schema: the mapping, the projection, and the refusals character for characterhooks/laya.hooks: the six events, composition, dispatch, the per-agent registry, process-wide defaults, andPolicywith its timeoutTypedDecisionsParityTesttyped-decisionscheckpoint driven end to end, which this PR previously left routable and unrunMODELS.md,CHANGELOG.md,samples/Twenty-one commits, each self-contained and green on its own, so this reviews commit by commit rather than as one diff. Two of them are fixes to code that is already on
main, described under "What the review of #927 found" below.Worth saying what the line count is, because it looks larger than it reads:
laya-java/scripts/--checked in CIKept as one PR rather than split per feature because the dependency runs
LanguageDetectionandreprintoRouter, so separate PRs would block on each other rather than review in parallel — and all three features edit the same generator, which would conflict three ways.Where Java is not Python, and what that cost
Every item below was found by a test that compares against the reference, and none of them would have failed a hand-written expectation.
Character.toLowerCaseis notstr.lower(). Beyond the contextual final-sigma rule, it follows the JDK's Unicode version: at 13.0 it does not know the lowercase of 40 code points CPython 3.12 maps, all of Vithkuqi among them, and returns them unchanged. The lowercase map is therefore a recorded table, which also makes a routing decision independent of the JDK a consumer happens to run — a JDK upgrade must not silently change which script a request is detected as. Same reasoning forisalpha(the JDK misses 4,863), the\wclass (4,883),\d(30),isupper(40),islower(200), and the canonical combining class, which the JDK does not expose at all and which the Mn/Mc categories differ from on 1,460 code points — including U+034F COMBINING GRAPHEME JOINER, whose class is zero.Character.isWhitespaceis notstr.isspace(). It rejects four of its members: U+0085, U+00A0, U+2007, U+202F.isSpaceCharis wrong the other way, accepting those while rejecting tab, newline and every control. Detection strips and splits on this set, so a no-break space pasted out of a word processor fuses two words into one token no stopword list holds — and NBSP is everywhere in real ticket text.String.format("%.0f", v)rounds halves up; CPython rounds to even. A route reports a percentage of letters, and one non-Latin letter in eight is exactly 12.5%, so the reason would read "13%" where the reference says "12%". Two route cases exist only for this: seventy Latin letters against ten Greek ones, and five accented characters in exactly forty.represcapes every unprintable code point, not just ASCII ones. One reason interpolates the mixed segment — a slice of the caller's own text — so this sees arbitrary input. Stopping at U+007F leaves U+00A0, U+00AD, U+200B, U+FEFF and the private-use and unassigned planes raw in an API response, while every printable non-ASCII character is deliberately left alone. That needed one more recorded table,str.isprintable().Insertion order is data, not formatting. It orders the public script profile and breaks every tie between two languages with the same score, so
LinkedHashMapthroughout. Thelang_tablesfixture was sorting the stopword order away, which meant it could not gate the tie-break at all; it now records that order explicitly.The four regexes are hand-written scanners, because Java's
\wadmits combining marks and join controls that Python's does not, andString.splitdrops trailing empty fields where Python's keeps them. The identifier scan reproduces the reference's lookbehind, which is what keeps it linear: without it a 4,000-character token costs 205 ms against 0.45 ms for prose, and a state is user input. Asserted structurally on 200,000 characters rather than with a stopwatch.The one place a tolerance was unavoidable, and what it cost
Everything in this PR is compared exactly except the shortlist's cosine scores, and that exception is measured rather than conceded. The reference scores with
np.dot(matrix, vector), a BLAS matrix-vector product, and how BLAS blocks that accumulation differs between implementations: on Apple Accelerate one row of a four-row product differs in the last bit from the same row computed alone. Every intermediate -- the dot, both norms, the division -- agrees bit for bit between numpy and a sequential loop. It is the blocking of the matrix form that does not, so the reference's own score is not reproducible across machines and no port can match it bit for bit.That has a second consequence worth flagging, because it would have shown up as a CI failure rather than as a parity caveat: recorded verbatim, these scores would make the fixture disagree with ITSELF between a developer's macOS and CI's OpenBLAS, and the drift gate would go red for a reason unrelated to any port. The fixture therefore records twelve decimals -- four orders of magnitude tighter than any porting error this repository has seen, four orders looser than the 1e-16 BLAS spread -- and regenerating twice now produces identical bytes. What stays exact is the LABEL ORDER, which is the only thing a consumer observes and the thing a ranking bug actually moves.
A test guards the tolerance from widening: it fails if the recorded precision and the comparison stop agreeing. And two rules that the tolerance would have hidden are asserted exactly instead, because a mutant proved it: a query whose norm underflows to zero scores 1.4e-200 without the zero-norm guard, and the self-cosine of
[1/3, 1/3, 8/3]computes to 1.0000000000000002 without the clamp. Both agree with the right answer to twelve decimals.Faster than the reference and than laya-ts
Same corpus, best of 7 after 3 warmups, one machine:
analysecallsTwo changes earn most of that and neither costs a divergence. Script lookup is a binary search over one sorted table instead of a scan over 25 scripts in order — equivalent because no two ranges overlap, which the generator now refuses to emit a table without, and which a test proves by sweeping every code point. And each character predicate answers ASCII from a pair of 64-bit masks computed from the same predicate as its range table. Counting letters by script tallies into an
int[]with no boxing, hashing or string comparison per character, where the reference does a dictionary update per non-Latin letter.One place this cannot copy the reference
In
laya, eviction drops a reference and CPython's refcounting keeps the agent alive for whoever still holds it. A JVM will not free a native ONNX session that way, so the router has to close it — and closing one a caller is mid-prediction with would crash that prediction. So eviction marks a slot retired and closes it when the last lease is released;predictleases internally, so the ordinary path is safe without a caller thinking about it. An agent handed in throughattachis never closed: the caller kept ownership.Loading goes through a factory seam rather than a hard-coded layout, because the reference downloads from the hub and this port takes local paths —
checkpointsRoot(path)for the obvious case,agents(factory)for anything else. A seam for the reasonInferenceSessionis one: the parts most likely to be wrong are which agent eviction picks, whether an attached one is ever closed, and whether two threads asking for the same cold checkpoint pay for two copies. Through a stub those run in milliseconds in every CI job, instead of only in the parity lane.How it is tested
1,351 tests pass with all three checkpoints and both ONNX graphs attached — 0 failed, 0 errored, and the only abort is
TestJvmVersionTest, which exists to be enabled by the matrix lane. 917 of them need no model at all (934 register, 17 abort by assumption on weights the machine does not have), and 918 run in the matrix lane, where-PtestJavaVersionis what enables that one extra test. Every model-backed class executes rather than aborting:PredictParityTest9,TypedDecisionsParityTest14,PredictLongTest49,DecoderParityTest40,SequenceParityTest52,TokenizerParityTest86,TokenDecoderTest195,LanguageDetectionParityTest116,WindowPlanTest66. Every expectation is recorded by running thelayapackage, never written by hand: 106 detection cases asserting all eight output fields and all fourteen intermediate steps, 47 route cases covering every precedence level and detection branch, the preset sets compared word for word, and — for the character tables — a sha256 over all 1,114,112 code points per property, so the tables are proven exhaustively rather than on the cases someone thought to list. That is what caught thetoLowerCasegap; a sampled corpus would not have.86 mutants were run against the suite; 82 die. The four survivors are each equivalent today for a reason that is now a guarded invariant rather than a coincidence, with a test that fails if the enabling property changes:
typed_decisionstask rewrite cannot change an answer, because name resolution lowercases first and the alias table already holds both spellings.The rounding mode needed the same treatment in reverse: it is unobservable through
analyseat an even threshold, so a half-up mutant passed the entire corpus. The rule is now pinned by a fixture and the unobservability itself asserted, so it fails the moment that threshold becomes odd.What reviewing the four new items found
Each of
predictLong, structureddecide, hooks andtyped-decisionswas put through anindependent adversarial review before being merged here, and all four reviews found real
defects. The pattern is more useful than the list: in every case the arithmetic
reproduced to the digit and the claim about what it proved was false. None of the defects
were wrong numbers; they were correct numbers above sentences that did not follow from them.
was compared. It ran ninth of thirteen — JUnit 5 orders nothing by default and the
class shares one agent, so whichever test runs first opens the weights. Real exposure to a
wrong checkpoint was 11 of 13 tests passing, not the disclosed 7 of 8, because all
three batch configurations passed too. And the clamp and bucket assertions had no
discriminating power at all:
temperature_by_optionsis byte-identical betweenenglishand
typed-decisions,choice:11+at 0.10058280825614929 included, so onlymax_len(1024 against 512) and
head_max_len(256 against 192) separate them. The config is nowasserted inside the shared accessor so it gates every consumer, and
unbucketed-scorewasadded — a six-level score question landing in
score:6-10, a bucket neither checkpointships, so the fitted
temperature[]triple finally reaches a forward pass. Under the samemispairing the suite now yields 4 tests, 3 failed, with both factories failing to register
at all. The mutant that proves it: nulling the per-type temperature fallback kills the new
case and survives the old fixture.
spot-check left six alive, and the "unkillable" one died to an eleven-line test — a
scope's removal direction is observable, because with
[a, x, a]backward removal leaves[a, x]and forward leaves[x, a], a different dispatch order. Three emphaticallydocumented contracts had no test at all: that an
Erroris never swallowed, that thetimeout thread is a daemon, and that
withoutDefaultHooksnests. The javadoc also claimedthe deadline "protects the request"; measured, an abandoned hook won 30 of 60 identical
calls, and
ctx.usageended up describing one result list whilectx.resultsheldanother. That hazard is reference-faithful — Python abandons the same thread holding the
same context — so the fix is a latch the reference cannot easily have, plus honest wording,
with the residual stated in the javadoc (2–17 of 60 at a colliding overrun) rather than
rounded to "fixed".
decideclaimed refusals reproduced character for character and 39 of 39mutants killed. Two mutants survived, and there was one outright correctness bug: a
non-finite
scorewas narrowed by(long) Math.rint(...), so on a field declaredminimum: 1, maximum: 3,+Infinityprojected to-9223372036854775808andNaNto1, silently, with nothing in theDecisionsaying so. CPython raises on both. SeparatelyNaN/Infinitycarriedjson.dumps's spelling rather than CPython'snan/inf, whichmatters past messages because the label is what the model is shown and what the answer
comes back keyed by.
build.gradle.ktskeeps both halves of its environment block, because one review found
LAYA_TYPED_ONNX_GRAPHmissing from the pass-through list and a different one found thatenvironment(...)is not a Gradle task input — so toggling a lane lefttestUP-TO-DATEand handed back the previous lane's results, a measurement of the wrong thingthat looks exactly like a measurement of the right one.
scripts/gen_fixtures.pyis mergedstructurally rather than textually, because git aligned the hunks on
}andtry:and cutthem mid-function, which does not parse; the two sides add 27 and 17 top-level blocks with
no name collision and both edit only
FAMILIES, so the result is their union at 18families. And the test floor is re-derived rather than reconciled, for the reason under
The CI lanes.
One further defect came from running CodeQL over the merged result rather than over any
branch: a high-severity
java/unreleased-lockinHooks.dispatch, which neither semgrepnor a 14-mutant sweep over that same file saw. The code was already
try/finally-paired —the defect is that
policy.lock()was called twice, once for each half.Policyis arecord, so both calls return the same monitor today and the path is unreachable; nothing in
the type requires that, and locking one instance while unlocking another leaves the first
held with no owner able to release it, in the one place where a stuck lock wedges every
hooked call rather than one. It is not a property of any line — it is a property of which
paths reach the unlock, which is why a pattern scanner cannot see it. Fixed by reading the
lock once into a local. CodeQL now reports 12 findings on this branch, none in the
blocking
security-extendedsuite; the remainder are pre-existing at the base commit orare
java/unused-parameteronHook's default no-op methods, where an unused contextparameter is what a no-op default is.
On the Python in this diff
laya-java/scripts/holds the generators. They run thelayapackage and record its output intolaya-java/fixtures/*.json, which is what the Java tests assert against — so the port is tested and benchmarked against the reference rather than against values I wrote myself.The SDK itself neither builds nor tests with Python: the fixtures and generated tables are committed, nothing in
src/starts a process, and the whole suite passes on a PATH with nopythonon it. Python is needed only to regenerate those files, and in CI to assert the committed copies still match the reference, so drift fails a check instead of going unnoticed. That gate covers every family it can reach without a checkpoint; the model-backed ones announce themselves as unverified rather than passing silently.The CI lanes
This lane had two jobs where the .NET lane has five, so it is now five as well, split along axes that are specific to this port rather than copied:
jvm build + model-free testsjvm model-free tests (jdk 21)/(jdk 24)jvm parity (tokenizer)jvm parity (predict)jvm parity (typed-decisions)jvm security scanThe two model-free floors are exact, with no margin, which is a change from the margin this PR used to carry. Four branches merged here each raised that floor against their own base — 779, 686/687 and 813/814 — and every one of those leaves over a hundred tests of slack against the merged 917, so each would have been decorative. A margin only binds classes larger than itself, and three classes in this lane run exactly one test, so no positive margin can bind them at all. Proved by deleting a class rather than by arithmetic: with
WindowPlanTestremoved the suite stays green at 916 run, the exact floor exits 1, and 813, 779 and 686 all exit 0. The two lanes differ by exactly one on purpose, for the reason above.The JDK matrix is the one this port needed, for the reason above:
\p{L}follows the JDK's own Unicode version, andCharacter.isLetteranswers 0 of 751 on Corretto 17 and 751 of 751 on Corretto 24 across the blocks this port has to classify. A lane that merely installs another JDK would not catch that, because the toolchain pins the compiler to 17 — Gradle would compile and test on 17 whatever was installed and the cell would go green having tested the same thing twice.-PtestJavaVersionmoves the test JVM only; the bytecode stays at class file major 61, verified while the tests ran on 24.0.2. Two guards keep it honest:launcherForwith auto-download off fails the build outright on a JDK the runner lacks instead of falling back to 17, andTestJvmVersionTestasserts the running JVM from inside the suite — necessary because Gradle writes<properties/>empty into the JUnit XML, so nothing downstream can tell 17 from 24.The parity split is by whether a cell needs the traced graph. Only one of the four checkpoint-backed test classes does; the rest need
tokenizer.jsonandrl_agent_config.json. The tokenizer cell is 183 tests in 8 seconds over 3 fixture families regenerated in 13, and it was waiting on a gigabyte-scale export it never reads — the export being the entire runtime of this lane, and also where the pre-tokenizer bug above was caught. It is deliberately not split per checkpoint the way the .NET lane is: those three fixture families each carry both checkpoints' sections in one document, so a job holding one of them regenerates none of the three and--strictfails by construction, and the Java tests assert both checkpoints' cases are present — a floor that exists because a@TestFactorywhose fixture key emptied used to generate zero assertions and still exit 0. Splitting per checkpoint would mean deleting that floor to buy nothing, since the two tokenizers together are megabytes.Both lanes stay advisory, like the .NET ones and for the same reason stated in that workflow's header.
The lanes were then reviewed as adversarially as the runtime, which was worth doing: three of the six findings made a cell green while measuring the wrong thing. Gradle ignores
org.gradle.java.installations.*passed as-D— withauto-detect=falseas-DthejavaToolchainsreport still listed all 8 local toolchains, and as-Pit dropped to 3 — so the JDK cell was passing by coincidence, on setup-java making the lastjava-versiontheJAVA_HOME.--allow-aborted, which that lane needs because the checkpoint-backed factories abort in it, also forgave the one@EnabledIfSystemPropertytest that proves the lane is not vacuous, so losing-PtestJavaVersionwould have run JDK 17 twice and reported two JDKs;check_test_results.pygrew--require-passedfor that. And the exported-graph cache key did not include the pinned revision — which lives inprepare_checkpoint.py, a file the key did not hash — while the script short-circuits on an existing graph, so a revision bump would have restored the old graph, re-recordedpredict.jsonfrom it, and agreed with itself. That one is now fixed at the source as well: the script stamps the revision beside what it produces and reuses it only on a match. The Hugging Face cache step was also a measured no-op (local_dirbypasses the hub cache entirely) and is removed rather than left looking like a saving.Also fixed here
A table-size cliff rather than waiting for it. With every table in its field initialiser the class stopped compiling with "code too large" once the lowercase map was added, because array literals share one
<clinit>and a JVM method body may not exceed 65,535 bytes. Each table now has its own factory method, and the generator refuses to emit one too large for that budget instead of leavingjavacto report it as a size error.Docs and samples
MODELS.mddocuments whatAgent.openactually takes — two directories that already exist — and the one script that makes both, with the--no-graphpath first, because most of what this SDK does needs no weights at all.CHANGELOG.mdis in Keep a Changelog form, and its Known limitations section is the point of it: what is not implemented, the 88-in-200,000 softmax divergence, the seven tokenizer behaviours no shipped checkpoint declares, and the seven places Java is not Python.Three samples — quickstart, routing and shortlisting, benchmark — which closes the one gap where
laya-javawas behind both siblings. They are single-file source programs rather than a Gradle module, deliberately: asamplessubproject would inherit the root build'sMavenPublicationandpublishAllPublicationsToCentralRepositorywould push it, which is why the emptylaya-java-clientmodule was taken out ofsettings.gradle.kts. They compile in CI with-Werror, so one that stops matching the API fails the build.All three were run, which found two things a compile could not:
Routingrouted an English state and would have asked a reader to export a second 1.3 GB graph, andBenchmarkwarmed up 20 times on a batch of 16 — about 340 forward passes on one line, which reads as a hang.typed-decisionsnow has recorded end-to-end fixtures, which is a change from the first version of this PR:fixtures/typed_decisions.jsonrecords the forward pass from the pinned revision — nine single-state cases spanning all four fine-tuned workflow signatures, three batch configurations, and two checks on the weights themselves. The routing decision and the forward pass are asserted on one object, throughRouter.matchTypedDecisionsWorkflow, instead of in two suites that never meet.Still not in this PR
The
laya-java-clientHTTP module and Android. Hooks,predictLongand structureddecidehave all landed here since this was opened. The README's status section lists what remains, and the CHANGELOG lists it under Known limitations rather than implying the port is complete.Three deliberate non-ports are documented rather than silently missing:
AsyncHook(it exists in the reference to finish a coroutine from synchronous code, and a JVM call is already synchronous), hooks onRouterandpredictLong(the reference'spredict_longsurvives hooks only via a start probe, a post-chain budget check and two separate "a hook answered" paths, none of which this port has — without them a hook that adds one option silently re-truncates every window), andPredictContext.router/decision, since a field that is always null is a promise a caller cannot tell from a bug.Email has landed.
laya.email's cleaner and state builder match the reference byte for byte on 93 recorded cases and on a 40,000-case differential sweep — 0 divergent — and the port is worth a note of its own, because the exact translation of one construct was unusable. A word boundary is defined in terms of\w, so translating\bfaithfully means a lookaround over Python's word class, which inlines 10,622 characters at 26 sites, makes a 277,309-character pattern, and runs at 1,742 µs per paragraph against 7.8 µs for Java's own\b— the entire cost of cleaning a message, 85× the reference. The two forms agreed on all 20,000 paragraphs of the sweep, which is why the slow one was not kept as the "safe" option: agreement on a corpus is not equivalence, and Java's\wis the JDK's. The boundary is still decided by the recorded table, just outside the pattern —boundaryViewrewrites the input so Java's word class and Python's agree on every character of it, and Java's fast\bthen finds Python's boundaries. End to end that is 0.029 ms per case against the reference's 0.026, warm and best-of-three on the same 4,000 bodies.Worth recording how close the corpus came to proving nothing: ten mutants were written against the port and six survived the first version of it — the combining-class table, the mark-without-a-base rule, the 40-character sign-off limit, the three-token tail, the attribution head,
UNIX_LINESand both space classes were all invisible to 83 passing cases. Eight cases were added to kill them and a ninth for a non-ASCII digit; all thirteen mutants now die.Separately: a
laya.langbug this corpus foundA line of bare acronyms inside an otherwise English state is named as French and routes the whole state to the multilingual checkpoint.
_named_prose_languageskips the_LETTER_RUNacronym guard when the segment holds no lowercase letter — the "a customer shouting in Portuguese is still Portuguese" case — so acronyms alone on their own line defeat it:"MON LA EST COM DES"scores French 4 to English 0 and clears the margin. Reproduces onmainwithlaya.lang.analyse. The port pins the current behaviour so it stays faithful.A fix now exists on
fix/lang-allcaps-acronyms, kept out of this PR because it changeslaya/lang.pyand belongs on its own. It is worth saying what reviewing it found, since the first attempt looked right: the obvious guard — blank the all-caps runs and re-vote — has no evidence bar at all, so when the words that named the language are the shouted ones, the evidence simply disappears. Measured over real MASSIVE rows,sag mir das HEUTIGE DATUMandquiero cancelar mi PEDIDO POR FAVORboth stopped being named, and 1,753 of 20,708 rows regressed with the last two words capitalised, 3,835 with the first half capitalised, and nothing improved. Monotonicity broke outright: 4,285 rows where the fully-upper-cased form routed multilingual but the same text with"ok "prefixed routed English — one lowercase English token making foreign detection strictly worse. Narrowing the guard to acronym-shaped runs cuts that to 447 and 856.The second thing that review found is about the evidence rather than the code. The no-degradation sweep reported a per-locale delta of exactly
0.000000across all 51 MASSIVE locales — because not one of its 151,674 rows contains a single uppercase character, so_shouted()is always false and the changed code is unreachable. A sweep that cannot observe the change is not evidence that the change is safe, and that is worth knowing before anyone trusts that harness for a case like this one.