Skip to content

fix(laya-ts): stop reading all-caps lines as French or Portuguese - #1336

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
thread3d:fix/laya-ts-acronym-routing
Oct 24, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
thread3d:fix/laya-ts-acronym-routing

Conversation

@thread3d

Copy link
Copy Markdown
Contributor

What

Ports Python's three pieces of caps handling into laya-ts/src/lang.ts: shouted / shoutedEvidence, the whole-state veto in analyseText, and the mixed-case blanking of acronym-shaped runs (only) when re-taking the verdict.

Why

Without them an acronym block voted as prose. Ordinary English support text -- Hello, / NAV COM OS 2 DESKTOP SUPPORT / I cannot connect to the VPN -- read as Portuguese and took the multilingual checkpoint, which router.py's own table puts 6-13 accuracy points behind English on English, and the same state answered differently from Python for the same input.

How it was verified

cd laya-ts && npx tsc -p tsconfig.json --noEmit
cd laya-ts && npx vitest run

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main (branch cut from upstream/main at 0.4.14)
  • Tests pass locally
  • Docs or examples updated when the public API changed

Python has three pieces of caps handling that this port never carried, all in
laya/lang.py:605-800:

  * _shouted / _shouted_evidence -- an all-caps segment keeps its tokens, but they
    only name a foreign language when one is a word (>= 5 letters) AND either the
    line carries non-English diacritics or a matched function word is >= 4 letters.
    Acronym-shaped tokens (MON, DES, EST, LA; NAV, COM, OS) are not evidence.
  * the whole-state veto in _analyse_text -- the same bar applied to the verdict, so a
    state that is only an acronym line is undecided rather than French.
  * the mixed-case blanking in _analyse_text -- re-take the verdict without the
    all-caps runs that elected the language, blanking only ACRONYM-SHAPED runs so
    emphasis capitals on the words that carry a sentence are left alone.

Without them, ordinary English support text with a technical block in capitals --
"Hello, / NAV COM OS 2 DESKTOP SUPPORT / I cannot connect to the VPN" -- read as
Portuguese and took the multilingual checkpoint, which router.py's own table puts
6-13 accuracy points behind English; and the same state answered differently from
Python for the same input.

Verified against the Python reference's own recordings: on the 246 states of
laya-rust/tests/golden/routing/lang_probe.json this port disagreed on 15 before and
6 after, the 9 that moved are exactly the shouted/acronym cases (shouted_acronyms,
shouted_acronyms_under_english, shouted_acronyms_two_english_lines, shouted_address,
shouted_german_ascii, shouted_german_umlaut, segment_acronym_line,
emphasis_acronym_mixed, leaf_shouted_acronyms), and nothing that agreed before
disagrees now. The 6 that remain are pre-existing divergences elsewhere (diacritic
rounding, leaf handling), untouched here.

The new tests use those recorded states plus the reported mail shape; ten of them
fail without this change.
@NandhaKishorM
NandhaKishorM merged commit 160fa8f into NandhaKishorM:main Oct 24, 2026
46 of 50 checks passed
NandhaKishorM added a commit that referenced this pull request Oct 24, 2026
33 PRs. Server hardening (#1331, #1332, #1333), TypeScript parity
(#1330, #1336, #1337, #1338, #1339, #1340, #1341, #1342, #1343),
fail-closed input handling in evals and the integrations (#1317, #1318,
#1319, #1320, #1321, #1322, #1326, #1335), exact and faster script
detection (#1334, #1349), JVM and .NET resource handling (#1327, #1328,
#1329, #1346), port goldens and clipping (#1324 excluded, #1325, #1344,
#1345), CI (#1323, #1347, #1348).

Verified before tagging: all 92 suites pass, 610 tests. ruff clean on
both lists. laya-ts 1247 tests across 44 files, tsc --noEmit clean.
Version agrees across pyproject.toml, laya.__version__ and nix/package.nix.
README carries 0.4.15 only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@thread3d
thread3d deleted the fix/laya-ts-acronym-routing branch October 28, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants