Repository navigation
fix(lang): describe the state in analyse, and round rates like Python - #1330
Merged
NandhaKishorM merged 1 commit intoOct 24, 2026
Merged
Conversation
`analyse` promises the script fields of the state it was given. When the
per-value pass flipped `isEnglish`, the TS runtime returned that leaf's
detection whole, so a 98.6% Latin ticket carrying one Russian line reported
`script: "cyrillic"` with `scriptProfile: {cyrillic: 1}` and
`nonLatinFraction: 1` -- three fields that contradict
`scriptProfile(stateText(state))` for the same input, and that a caller
reading `analyse(state).script` cannot tell apart from the `isEnglish` flag
beside them. The override now answers `language` and `isEnglish` only, as
Python's `analyse` does.
The reported rates also broke a tie away from zero, where Python's `round()`
takes the even digit: one non-English letter in 32 characters is 0.03125
exactly, reported as 0.0313 here and 0.0312 by the reference. Scaling by
10000 also invents a tie that the double is not -- 1 in 4000 is
0.0002500000000000000052, which Python rounds up and a `* 10000` tie rule
rounds down -- so the digits come from the double's own expansion.
Differential-tested against Python's `laya.lang.analyse` over 6 616 string
states and 1 504 structured ones: the structured disagreements drop from
108 to 0 (the two remaining rows in that run feed raw bytes through a JSON
harness), the string ones from 6 to 1. The one left is the all-caps bar,
which the port does not have; not addressed here.
NandhaKishorM
added a commit
that referenced
this pull request
Oct 24, 2026
fix(laya-ts): stop reading all-caps lines as French or Portuguese Add/add conflict against #1330, which had landed a comment documenting round4 at the same point in lang.ts. Both sides are wanted and unrelated, so this is a union: nonLatinCarries goes first because analyseText calls it, and #1330's comment stays attached to the round4 it explains. NON_LATIN_FRACTION, NON_LATIN_MIN_FRACTION and NON_LATIN_MIN_LETTERS were already defined above. tsc --noEmit is clean and the laya-ts suite passes 1247 tests across 44 files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
NandhaKishorM
added a commit
that referenced
this pull request
Oct 24, 2026
33 PRs. Server hardening (#1331, #1332, #1333), TypeScript parity (#1330, #1336, #1337, #1338, #1339, #1340, #1341, #1342, #1343), fail-closed input handling in evals and the integrations (#1317, #1318, #1319, #1320, #1321, #1322, #1326, #1335), exact and faster script detection (#1334, #1349), JVM and .NET resource handling (#1327, #1328, #1329, #1346), port goldens and clipping (#1324 excluded, #1325, #1344, #1345), CI (#1323, #1347, #1348). Verified before tagging: all 92 suites pass, 610 tests. ruff clean on both lists. laya-ts 1247 tests across 44 files, tsc --noEmit clean. Version agrees across pyproject.toml, laya.__version__ and nix/package.nix. README carries 0.4.15 only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two fields of
analyse()'s result described something other than the state it was given. Both are fixed inlaya-ts/src/lang.ts, with a newlaya-ts/tests/state-script-fields.test.ts.1. A per-value override replaced the state's script fields with the leaf's. When the value pass flips
isEnglish, the result now keeps the state'sscript,scriptProfile,diacriticRateandnonLatinFraction, and overrides onlylanguage,isEnglishandlanguageUndecided— the same three Python'sanalyseoverrides.2. The reported rates rounded a tie away from zero.
round4now rounds the double's own decimal expansion, so a tie goes to the even digit as Python'sround()does, and a value that only looks like a tie after* 10000is not treated as one.Why
analysereturns the script fields of the state. They were coming from one leaf instead:The state is 98.6% Latin, and
scriptProfile(stateText(state))already said so for the same input, soanalysedisagreed with the module's own exported function. Anything readinganalyse(state).scriptor.nonLatinFraction— a "detected language" label, a per-script metric — was told the ticket was entirely Cyrillic, and theisEnglish: falsebeside it does not reveal that the route came from one field.The rates: one non-English letter in 32 characters is
0.03125exactly.Math.round(0.03125 * 10000) / 10000is0.0313. The obvious fix — a half-to-even branch onx * 10000— is wrong in the other direction: 1 in 4000 is0.0002500000000000000052as a double, which Python rounds up to0.0003, while0.00025 * 10000lands on2.5exactly and a tie rule would report0.0002. Both directions are pinned by tests.How it was verified
cd laya-ts npm install npx tsc -p tsconfig.json npx vitest run44 files / 1224 tests passed. The new file fails 7 of its 9 tests onmainand passes all 9 here (git stash push laya-ts/src/lang.tsthen re-run).Differentially against Python's
laya.lang.analyse, over 6 616 string states and 1 504 structured ones (nested dicts/lists, 11 languages and scripts, a 4 000-character cap, byte inputs):Every disagreement in that run was a field in this PR's scope. The one left is
MON DES EST LA: Python has an all-caps bar (_shouted_evidence) that the port does not have, so it reports the state as undecided-English where the port names French. That is a routing change rather than a field-description bug, so it is not in this PR — happy to do it separately if you want the port to match there too.Checklist
analysereturning a field that does not describe the state; I can split them into two PRs if you would rather review the rounding on its own)main