Skip to content

fix(cli): --model resolves core's aliases instead of matching three names exactly - #546

Merged
NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:feat-cli-model-aliases
Sep 27, 2026
Merged

NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:feat-cli-model-aliases

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

fix(cli): --model took three spellings; core has thirteen

The gap

laya/cli.py:50 was:

parser.add_argument("--model", choices=sorted(laya.DEFAULT_MODELS), ...)

choices= is derived from core, so the names never went stale — but argparse matches a
choice exactly. Meanwhile the flag's whole job is to hand a value to
router.predict(model=...) / router.route(model=...), and those resolve the argument through
laya.router.normalise_name: trim, lowercase, then _ALIASES. Core has ten aliases for its three
checkpoints. The CLI accepted none of them, and its error message told the user those three
spellings were all that exist:

$ laya "I was charged twice, please refund" --model en
laya: error: argument --model: invalid choice: 'en' (choose from english, multilingual, typed-decisions)

--model en is what the README's own SDK examples say to write, and laya --lang de --model ml
is a normal thing to try. laya --predict --model en (loads the checkpoint) failed the same way,
so the only workaround was spelling the checkpoint out.

This is the same drift #544 fixed on the MCP surface, through a different mechanism: not a copied
set that went stale, but exact matching where core does normalisation.

Before and after (18 spellings, argparse only)

before after
accepted out of 18 3 18
accepted english, multilingual, typed-decisions those, plus en, laya, default, multi, ml, laya-multilingual, typed, typed_decisions, decisions, laya-typed-decisions, EN, " Multilingual ", "Typed-Decisions" — all canonicalised — and auto / AUTO, which pin nothing
refused 15 same as ever: gpt4, "", english-ish, laya-typed, and any non-string

The error text is core's, so it now lists both the checkpoints and the aliases, with
, or 'auto' appended:

$ laya "…" --model gpt4
laya: error: argument --model: unknown model 'gpt4'; choose one of ['english', 'multilingual',
'typed-decisions'] (or an alias: ['decisions', 'default', 'en', 'laya', 'laya-multilingual',
'laya-typed-decisions', 'ml', 'multi', 'typed', 'typed_decisions']), or 'auto'

With real weights (CPU, laya --predict --json)

One refund ticket through the router's own question set, five spellings of the English checkpoint:

--model exit routing.model domain difficulty is_sensitive needs_tools
english 0 english chitchat 1.354 0.88 0.001
en 0 english same same same same
laya 0 english same same same same
default 0 english same same same same
EN 0 english same same same same

0 decision changes — the alias path and the name path are the same call. The routing path
(no download) was checked the same way: --model en/ml/typed/auto each report their checkpoint
and explicit model='…' as the reason, and --model auto prints English Latin text, i.e. it is
exactly the same as leaving the flag off.

What changed

  • laya/cli.py — a model_name() type function on the flag: auto (any casing) returns None,
    which is what omitting the flag already means, otherwise laya.router.normalise_name decides.
    metavar="NAME" so laya --help reads --model NAME instead of enumerating three options.
    normalise_name is imported inside the function, matching how the rest of the CLI reaches the
    router lazily. One example line added to the module docstring.
  • README.md — the ### Command line block gains laya "My payment failed twice" --model ml.
  • tests/test_cli.py — 38 → 69 checks. The sweep imports DEFAULT_MODELS and _ALIASES from
    core rather than writing the names out, so a new checkpoint or alias is covered the day it lands
    instead of exposing the same gap again. Also covers: canonicalisation per spelling, the four
    rejected inputs, the error message still naming every option, and run_cli forwarding —
    --model en reaches router.route as "english", --predict --model ML reaches
    router.predict as "multilingual", --model auto reaches it as None.

Gates: ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120 clean,
compileall clean, tests/test_cli.py 69/69, tests/test_hooks_api.py 131/131.

Overlap

build_parser declared `--model choices=sorted(laya.DEFAULT_MODELS)`, and argparse
matches choices exactly. So the flag knew three spellings while the call it feeds,
`router.predict(model=...)` / `router.route(model=...)`, resolves core's whole
registry: three checkpoints plus ten aliases, case- and whitespace-insensitively.
`laya "..." --model en` exited 2 with "invalid choice" instead of pinning English.

The flag now runs its argument through `laya.router.normalise_name` -- the same call
core makes -- so it cannot fall behind the registry again, the name handed to the
router is already canonical, and `auto` means the same as omitting the flag.
@NandhaKishorM
NandhaKishorM merged commit a4986f6 into NandhaKishorM:main Sep 27, 2026
22 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Thanks @aashish254, merged. It ships in 0.3.21.

NandhaKishorM pushed a commit that referenced this pull request Sep 29, 2026
… resolves one

`laya/router.py:112 normalise_name` is core's one model resolver: trim, lower-case, map
`_ALIASES`, then check `DEFAULT_MODELS`. Every other surface hands a caller's `model`
string to it -- `laya/serve.py::_resolve_model`, `laya/cli.py::model_name` (#546),
`laya/mcp/tools.py` (#544) -- and `examples/server.py` was the exception: it tested the
stripped string for membership in a tuple of the three checkpoint names. So
`laya --model laya` pinned the English checkpoint while the same value POSTed to
/predict answered 422, and the caller's only notice was a validation error.

Measured on main, weight-free, 82 spellings probed against both resolvers: core accepts
67, the demo accepts 9. The 60 the demo refused are every alias (`laya`, `en`, `ml`,
`multi`, `typed`, `typed_decisions`, `decisions`, `default`, `laya-multilingual`,
`laya-typed-decisions`), every casing (`ENGLISH`, `Laya-Multilingual`) and every padded
form (`" default "`, `" ml "`). Over HTTP `{"model":"laya"}`, `{"model":"LAYA"}`,
`{"model":"ml"}`, `{"model":"laya-multilingual"}`, `{"model":" default "}`,
`{"model":"typed_decisions"}` and `{"model":"decisions"}` each answered 422 against a
Router that would have answered any of them.

`_check_model` now delegates to `normalise_name` and returns the canonical checkpoint, so
what reaches the Router and what the response reports is the checkpoint rather than the
spelling that named it. Blank stays "do not pin": the playground posts an empty `model`
field and core's resolver raises on one, so the demo keeps its own leniency there instead
of becoming stricter than core -- those two blanks are the only remaining difference
between the two resolvers, and they are deliberate.

Everything this server publishes about the field is read from the same two tables, so an
alias cannot outlive its target and a checkpoint core gains is pinnable here with no edit
to this file: `MODEL_ALIASES` is `_ALIASES` inverted and filtered to live targets, `/models`
reports the aliases it never listed, both request models' field text is rendered from the
lists (`BatchRequest.model` had no description at all), and the models page's checkpoint
count and lead text follow the payload. README's section for this server says the field
exists and where its spellings come from; before this the only way to learn it was to get
a 422.

Gates: `tests/test_example_server_limits.py` 66 -> 81 weight-free checks, and the probe
list is built from `laya.router.DEFAULT_MODELS`/`_ALIASES` rather than typed here, so it
covers spellings this file has never heard of. Two arms hold the derivation: one registers
a fourth checkpoint and an alias, reloads the demo, and requires it to be pinnable by name,
by alias, in the OpenAPI text and on `/models`; the other parses both lists back out of the
rendered description and compares them with what the router accepts, which a hand-typed
sentence would not survive. `tests/test_server_example.py` pins `/models`' key set, and the
payload gained a key, so that one assertion now lists `aliases` -- additive, no existing
key changed name or meaning.

Verified against real weights on MPS: 11 spellings across two checkpoints, every one HTTP
200 with an identical decision per checkpoint (department "billing", urgency 1.599 and
1.003), the reported checkpoint exactly {multilingual} and {english}, and blank still
agreeing with omitted. The 13-mutant battery (exact-match restored, spelling returned
instead of canonical, blank raising, alias map emptied, lower-only, literal description,
undocumented batch field, aliases dropped from the payload, typed page count, omit-hint
gone, README paragraph dropped, no reload, hardcoded probe) is caught by at least one
check per mutant and restores byte-exact. Full CI script list, 60 suites, reruns clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants