From 686330b2fdc51c20423034f3c9c83d4701119112 Mon Sep 17 00:00:00 2001 From: Bruce-Yii <298228875+Bruce-Yii@users.noreply.github.com> Date: Fri, 25 Sep 2026 02:55:54 +0800 Subject: [PATCH 1/3] docs: add CLI and MCP server guide --- docs/cli-mcp.md | 205 ++++++++++++++++++++++++++++++++++++++++++++++++ docs/index.md | 3 +- 2 files changed, 207 insertions(+), 1 deletion(-) create mode 100644 docs/cli-mcp.md diff --git a/docs/cli-mcp.md b/docs/cli-mcp.md new file mode 100644 index 000000000..55e16670d --- /dev/null +++ b/docs/cli-mcp.md @@ -0,0 +1,205 @@ +# Command line and MCP server + +Laya has two local interfaces for trying the same structured-decision engine: + +| Interface | Use it for | Transport | +|---|---|---| +| `laya` | quick checks and interactive exploration from a terminal | command line | +| `laya-mcp-server` | connecting an MCP client or agent to Laya's built-in tools | MCP over stdio | + +Choose the CLI when you are the person reading the result. Choose MCP when another process needs +a stable tool interface. Both use Laya's `Router` to select a checkpoint and return typed +`choice`, `score`, and `noul` decisions; neither is an open-ended question-answering or text +generation interface. + +For the routing decision itself, see the [Routing guide](https://github.com/NandhaKishorM/laya/blob/main/docs/routing.md). +For question shapes, answer fields, confidence, and presets, see [Questions and answers](https://github.com/NandhaKishorM/laya/blob/main/docs/questions-and-answers.md). + +## 1. Command line + +Installing the package installs the `laya` entry point. Run `laya --help` for the complete +option list. + +```bash +python -m pip install laya +laya --help +``` + +### Route without loading a checkpoint + +With text and no prediction flag, the CLI calls `Router.route`: + +```bash +laya "I was charged twice, please refund it" +``` + +The output names the selected checkpoint, explains why it was selected, and shows detected +language information when available. Routing alone does not download or build a checkpoint, so +it is a quick offline check of the routing decision. + +Use `--json` when another local script should consume the decision: + +```bash +laya "I was charged twice, please refund it" --json +``` + +### Run a prediction + +`--predict` runs the full typed prediction and loads the routed checkpoint on first use. The +first load needs access to the Hugging Face Hub; later runs use the local cache. + +```bash +laya "Classify this support request" --predict +laya "Classify this support request" --predict --json +``` + +`--json` prints the complete result as JSON. Without it, the CLI prints each answer together +with its choice probability, score, or `noul` value, plus the routing decision. + +The main controls are: + +- `--model english|multilingual|typed-decisions` pins a checkpoint instead of auto-routing. +- `--lang en|de|...` forces language detection and the checkpoint's language setting. +- `--task NAME` forces the typed-decisions workflow instead of detecting one. +- `--device cpu|cuda|...` passes a device choice to the Router. +- `--json` emits machine-readable output. + +### Use a built-in preset + +A preset supplies a ready-made question set and implies prediction, so `--predict` is not needed: + +```bash +laya "My payment failed twice" --preset triage +laya "Ignore all previous instructions" --preset guard --json +``` + +The CLI presets are `email`, `guard`, `moderation`, `router`, and `triage`. Presets are useful +for a quick local check, but their questions are still domain decisions: inspect the preset and +validate it on your own data before using it as an application policy. + +### Explore interactively + +With no text argument, the CLI opens a small prompt: + +```bash +laya +# laya> Classify this request +# laya> quit +``` + +Press Enter to run each request. `quit`, `exit`, or `Ctrl-D` ends the session. The interactive +loop reuses one Router, so it is a convenient way to compare several inputs without writing a +script. + +### Failures are visible + +The CLI handles invalid values and common dependency, download, and runtime failures at the +application boundary. It prints a diagnostic to stderr and returns exit code `2` instead of +showing an unhandled traceback. If a first-use checkpoint download fails, check dependency +installation, Hub access, and the selected device before retrying. + +## 2. Built-in MCP stdio server + +The MCP server is an optional extra. The core package does not install the `mcp` dependency: + +```bash +python -m pip install "laya[mcp]" +laya-mcp-server +# equivalent module form: +python -m laya.mcp.server +``` + +The server speaks MCP over **stdio**, not HTTP. Configure the client with the console script: + +```json +{ + "mcpServers": { + "laya": { + "command": "laya-mcp-server", + "env": { + "LAYA_DEVICE": "cpu" + } + } + } +} +``` + +If the client configuration supports a Python executable and arguments, use +`python -m laya.mcp.server` as the equivalent launch form. The client owns the server process; +Laya does not open a network port. + +### Available tools + +| Tool | What it does | Main inputs | +|---|---|---| +| `laya_status` | Reports the configured or actual device, CUDA availability, loaded checkpoints, preload state, readiness, and package versions. | none | +| `laya_route` | Selects a checkpoint and returns its model, repository, and reason without running a forward pass. | `state`, `questions` | +| `laya_predict` | Runs typed questions and returns answers, routing metadata, latency, and the answering device when readable. | `state`, `questions`, optional `model` (`auto`, `english`, `multilingual`, or `typed-decisions`) | +| `laya_preset` | Runs a built-in workflow using its built-in question set. | `preset`, `state` | + +`state` must be a non-empty JSON object. `questions` must be a non-empty object whose values use +Laya's typed question schema. `laya_preset` accepts `guard`, `moderation`, `triage`, and +`model_router`; the last name is the MCP spelling of the router workflow. The CLI's preset +names are listed separately above because the two entry points expose different preset aliases. + +A prediction call has the same shape as the SDK's typed call: + +```json +{ + "state": { + "body": "I was billed twice for the same plan. Please reverse the duplicate charge." + }, + "questions": { + "department": { + "type": "choice", + "instructions": "Which team should handle this request?", + "criteria": { + "billing": "payments, invoices, refunds, duplicate charges", + "technical": "bugs, outages, integration problems" + } + }, + "urgent": { + "type": "noul", + "instructions": "Does the user need immediate help?" + } + } +} +``` + +The tool response is JSON containing the typed `answers`, the `routing` decision, and timing +information. Do not treat a high-confidence answer as permission to perform an external action; +the application or agent remains responsible for policy, review, and side effects. + +### Startup and environment + +The MCP server keeps a resident Router and serializes first-time construction. By default it +preloads `english` and `multilingual`; `typed-decisions` stays lazy. A preload failure is +reported at startup and retried on the next tool call, so inspect `laya_status` before assuming +the server is ready. + +| Variable | Default | Meaning | +|---|---|---| +| `LAYA_DEVICE` | automatic | Device value passed to PyTorch, such as `cpu` or `cuda`. | +| `LAYA_PRELOAD` | `1` | Build the configured checkpoints at startup. Set to `0` for lazy loading. | +| `LAYA_MODELS` | `english,multilingual` | Comma-separated checkpoints to preload. An empty value keeps the MCP default rather than preloading every checkpoint. | +| `LAYA_THREADS` | PyTorch default | Caps Torch intra-op threads for CPU inference; keep it at or below the physical core count. | + +The environment names and meanings follow Laya's serving conventions. They configure model +lifecycle, not application permissions. The client still decides when to call a tool and what to +do with the returned decision. + +## 3. Shared boundaries and related guides + +The CLI and MCP server are interfaces to the same typed decision engine: + +- Use `choice` for a finite label set, `score` for an ordered rubric, and `noul` for a calibrated + probability of true. +- Validate thresholds and presets on representative data; there is no universal adoption + threshold. +- Keep irreversible or high-cost actions behind the application's review and fallback policy. +- The MCP server calls `Router.predict`, so configured prediction hooks still fire. See + [Prediction hooks](hooks/index.md), [hook lifecycle](hooks/lifecycle.md), and + [Tracing](hooks/tracing.md) for observability and `run_id` correlation. + +This guide covers the local CLI and the built-in MCP stdio server. It does not document the +HTTP API, community wrappers, or an MCP protocol redesign. diff --git a/docs/index.md b/docs/index.md index 5af631e4d..654f0cf24 100644 --- a/docs/index.md +++ b/docs/index.md @@ -22,6 +22,7 @@ is one copy to keep current. | log, redact, cache or gate every decision without forking Laya | [Prediction hooks](hooks/index.md) | | adopt Laya incrementally without granting execution permission | [Staged adoption](staged-adoption.md) | | route, screen or triage inside a LangChain or LangGraph app | [LangChain & LangGraph](langchain.md) | +| run Laya from a terminal or connect its built-in MCP stdio server | [Command line and MCP server](cli-mcp.md) | | run the SDK or the `laya-serve` HTTP API in a container, on CPU or an NVIDIA GPU | [Docker quickstart](docker.md) | | build for ARM64 hosts or DGX Spark | [ARM64 and DGX Spark containers](docker-platforms.md) | | specialise a checkpoint for your own decisions | [Browser-agent fine-tuning example](finetune_browser_agent.md) and the [fine-tuning notebook](https://github.com/NandhaKishorM/laya/blob/main/notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynb) | @@ -29,5 +30,5 @@ is one copy to keep current. | look up a class, function or parameter | [Python API reference](reference/index.md) | | score a labelled dataset, compare to a baseline, or gate a build on it | [Evaluation harness](evals.md) | -Routing, the HTTP API, the command line, the MCP server and confidence gating are in the +Routing, the HTTP API and confidence gating are in the [README](https://github.com/NandhaKishorM/laya#readme) for now. From bc1734fb4cbc826a0cb8085b2fc928b4b0bb3ba1 Mon Sep 17 00:00:00 2001 From: Bruce-Yii <298228875+Bruce-Yii@users.noreply.github.com> Date: Fri, 25 Sep 2026 03:12:12 +0800 Subject: [PATCH 2/3] docs: clarify CLI language override and links --- docs/cli-mcp.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/cli-mcp.md b/docs/cli-mcp.md index 55e16670d..9aefaf6b2 100644 --- a/docs/cli-mcp.md +++ b/docs/cli-mcp.md @@ -12,8 +12,8 @@ a stable tool interface. Both use Laya's `Router` to select a checkpoint and ret `choice`, `score`, and `noul` decisions; neither is an open-ended question-answering or text generation interface. -For the routing decision itself, see the [Routing guide](https://github.com/NandhaKishorM/laya/blob/main/docs/routing.md). -For question shapes, answer fields, confidence, and presets, see [Questions and answers](https://github.com/NandhaKishorM/laya/blob/main/docs/questions-and-answers.md). +For the routing decision and typed-question examples, see the README's [Route Mode quickstart](https://github.com/NandhaKishorM/laya#quickstart-route-mode-recommended). +For confidence and built-in workflows, see the README's [confidence gating](https://github.com/NandhaKishorM/laya#automated-confidence-gating) and [workflow presets](https://github.com/NandhaKishorM/laya#built-in-workflow-presets). ## 1. Command line @@ -59,7 +59,7 @@ with its choice probability, score, or `noul` value, plus the routing decision. The main controls are: - `--model english|multilingual|typed-decisions` pins a checkpoint instead of auto-routing. -- `--lang en|de|...` forces language detection and the checkpoint's language setting. +- `--lang en|de|...` supplies an explicit language code instead of automatic detection. - `--task NAME` forces the typed-decisions workflow instead of detecting one. - `--device cpu|cuda|...` passes a device choice to the Router. - `--json` emits machine-readable output. From becd5e40b318eb57ae631e6bbe4e3c0c2bf2096d Mon Sep 17 00:00:00 2001 From: Bruce-Yii <298228875+Bruce-Yii@users.noreply.github.com> Date: Mon, 28 Sep 2026 05:00:09 +0800 Subject: [PATCH 3/3] docs: bring the CLI and MCP guide onto current main MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rebases onto current main and re-verifies the page against the code it describes, then fixes four statements that current main no longer supports. The rebase needed care. The previous head was a merge commit, and the maintainer's requested corrections lived in that merge's hand-resolved conflict resolution rather than in an ordinary commit. A plain rebase drops merges, so replaying the branch produced a page that silently lost all three of them. The guide is restored with those corrections intact, plus the `laya_shortlist` and `laya eval` sections. Corrected against the code: * `laya_predict_batch`, `laya_route_batch` and `laya_decide` are registered by the MCP server but were missing from the "Available tools" table, which listed five of the eight tools. The 0.3.21 sweep put batching and the token budget on every surface; this page still described the pre-batch set. * `LAYA_AUTO_TASK` is read by the MCP server (`auto_task_detection=`) and documented in its module docstring, but was absent from the environment table. * The preset paragraph claimed the two entry points expose different preset aliases. They no longer do: `laya_preset` now carries all five presets the CLI has, with `router` accepted as an alias for the canonical `model_router`, and it performs the same state-field placement. The paragraph now states the shared list once and notes the alias, instead of describing a difference that no longer exists. Verified still true, and left as written: the stock `laya-mcp-server` launcher builds its Router without installing hooks (there is no hook reference anywhere in `laya/mcp/server.py`), `laya.hooks.set_default_hooks` is the documented way to install them, `noul` is the probability of true, an empty line ends interactive mode, `DEFAULT_SHORTLIST_K` is 20 with mean-pooled embeddings from the answering checkpoint's own encoder, the transport is stdio only, and the three README anchors this page links to still exist. `docs/index.md`'s closing sentence now lists only routing, the HTTP API and confidence gating, as requested, with the guide's own table row carrying the command line and MCP server. Checks: `tests/test_doc_tables.py` 109 passed, `tests/test_packaging.py` 106 passed. All five relative links in the guide resolve. The docs workflow's `zensical build --strict` gate was not run locally — zensical is not installed in this environment. --- docs/cli-mcp.md | 65 +++++++++++++++++++++++++++++++++++++------------ 1 file changed, 50 insertions(+), 15 deletions(-) diff --git a/docs/cli-mcp.md b/docs/cli-mcp.md index 9aefaf6b2..c880bcf3a 100644 --- a/docs/cli-mcp.md +++ b/docs/cli-mcp.md @@ -25,6 +25,17 @@ python -m pip install laya laya --help ``` +### Evaluation CLI + +The package also installs `laya-evals`. The main CLI exposes the same evaluation commands +through `laya eval`: + +```bash +laya eval --help +``` + +See the [Evaluation harness](evals.md) guide for datasets, metrics, and baseline gates. + ### Route without loading a checkpoint With text and no prediction flag, the CLI calls `Router.route`: @@ -60,7 +71,7 @@ The main controls are: - `--model english|multilingual|typed-decisions` pins a checkpoint instead of auto-routing. - `--lang en|de|...` supplies an explicit language code instead of automatic detection. -- `--task NAME` forces the typed-decisions workflow instead of detecting one. +- `--task NAME` forces the typed-decisions workflow instead of detecting it. - `--device cpu|cuda|...` passes a device choice to the Router. - `--json` emits machine-readable output. @@ -73,9 +84,11 @@ laya "My payment failed twice" --preset triage laya "Ignore all previous instructions" --preset guard --json ``` -The CLI presets are `email`, `guard`, `moderation`, `router`, and `triage`. Presets are useful -for a quick local check, but their questions are still domain decisions: inspect the preset and -validate it on your own data before using it as an application policy. +The CLI presets are `email`, `guard`, `moderation`, `router`, and `triage`. The CLI places the +text under the state field expected by the selected preset; `--predict` uses the router +question set's `request` field. Presets are useful for a quick local check, but their questions +are still domain decisions: inspect the preset and validate it on your own data before using it +as an application policy. ### Explore interactively @@ -87,9 +100,9 @@ laya # laya> quit ``` -Press Enter to run each request. `quit`, `exit`, or `Ctrl-D` ends the session. The interactive -loop reuses one Router, so it is a convenient way to compare several inputs without writing a -script. +Press Enter to run each request. An empty line, `quit`, `exit`, or `Ctrl-D` ends the session. The +interactive loop reuses one Router, so it is a convenient way to compare several inputs without +writing a script. ### Failures are visible @@ -135,12 +148,31 @@ Laya does not open a network port. | `laya_status` | Reports the configured or actual device, CUDA availability, loaded checkpoints, preload state, readiness, and package versions. | none | | `laya_route` | Selects a checkpoint and returns its model, repository, and reason without running a forward pass. | `state`, `questions` | | `laya_predict` | Runs typed questions and returns answers, routing metadata, latency, and the answering device when readable. | `state`, `questions`, optional `model` (`auto`, `english`, `multilingual`, or `typed-decisions`) | +| `laya_shortlist` | Shortlists a many-option choice question, then answers it and returns the shortlist metadata. | `state`, `questions`, optional `model`, optional `k` (default `20`) | | `laya_preset` | Runs a built-in workflow using its built-in question set. | `preset`, `state` | +| `laya_predict_batch` | Answers many requests in one call. Requests are routed first and grouped by checkpoint, so matching question schemas share forward passes; answers come back in input order. | `requests`, each `{state, questions, model?, task?, lang?}`, optional `batch_size` | +| `laya_route_batch` | Decides which checkpoint would answer each request, with no forward pass and no checkpoint load. | `requests`, same shape as `laya_predict_batch` | +| `laya_decide` | Answers a JSON-schema-shaped decision in one forward pass and returns the decided values with per-field confidence, instead of an answer map to parse. Schema properties may be enum choices, booleans, or integers with a minimum and maximum; free strings, arrays, and nested objects are rejected by path. | `state`, `schema`, optional `model` | + +The three batch and schema tools exist because the same operations are available on the SDK and +`laya-serve`: reaching for many requests, or for a caller that already knows the answer shape, +does not require dropping to Python. For the schema-driven form in more depth, see +[Schema-driven decisions](structured.md). + +The shared guardrail says not to send choice questions with more than 20 options without +shortlisting. `laya_shortlist` keeps the `k` most likely labels before the forward pass; its +default is `k=20`. It uses mean-pooled embeddings from the answering checkpoint's own encoder, +so it does not download a second model, and returns the kept labels, cosine scores, `k`, and +option count for each shortlisted question. `state` must be a non-empty JSON object. `questions` must be a non-empty object whose values use -Laya's typed question schema. `laya_preset` accepts `guard`, `moderation`, `triage`, and -`model_router`; the last name is the MCP spelling of the router workflow. The CLI's preset -names are listed separately above because the two entry points expose different preset aliases. +Laya's typed question schema. `laya_preset` accepts the same five presets the CLI does: `email`, +`guard`, `moderation`, `triage`, and the router workflow, whose canonical name on this surface is +`model_router`. `router` is accepted as an alias and names the same preset, so the CLI spelling +works here too; the canonical key is the one that comes back in the result. Given a state of +exactly one string, `laya_preset` places it under the field that preset's questions name, the +same placement the CLI does, so a caller does not have to guess the key. Anything richer than one +string is the caller's own shape and is passed through untouched. A prediction call has the same shape as the SDK's typed call: @@ -183,21 +215,24 @@ the server is ready. | `LAYA_PRELOAD` | `1` | Build the configured checkpoints at startup. Set to `0` for lazy loading. | | `LAYA_MODELS` | `english,multilingual` | Comma-separated checkpoints to preload. An empty value keeps the MCP default rather than preloading every checkpoint. | | `LAYA_THREADS` | PyTorch default | Caps Torch intra-op threads for CPU inference; keep it at or below the physical core count. | +| `LAYA_AUTO_TASK` | `0` | Set to `1` to let a request auto-route to the `typed-decisions` checkpoint. Same meaning as in `laya.serve`; it does not preload that checkpoint, so `LAYA_MODELS` still decides what is built at startup. | -The environment names and meanings follow Laya's serving conventions. They configure model -lifecycle, not application permissions. The client still decides when to call a tool and what to -do with the returned decision. +The stock `laya-mcp-server` launcher creates its Router without installing hooks. If you need +prediction hooks, use a custom launcher that installs them, for example with +`laya.hooks.set_default_hooks`, before the server builds its Router. The environment variables +above configure model lifecycle, not hook registration. The client still decides when to call a +tool and what to do with the returned decision. ## 3. Shared boundaries and related guides The CLI and MCP server are interfaces to the same typed decision engine: -- Use `choice` for a finite label set, `score` for an ordered rubric, and `noul` for a calibrated +- Use `choice` for a finite label set, `score` for an ordered rubric, and `noul` for the probability of true. - Validate thresholds and presets on representative data; there is no universal adoption threshold. - Keep irreversible or high-cost actions behind the application's review and fallback policy. -- The MCP server calls `Router.predict`, so configured prediction hooks still fire. See +- The MCP server calls `Router.predict`, so hooks fire when a custom launcher installs them. See [Prediction hooks](hooks/index.md), [hook lifecycle](hooks/lifecycle.md), and [Tracing](hooks/tracing.md) for observability and `run_id` correlation.