Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
240 changes: 240 additions & 0 deletions docs/cli-mcp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,240 @@
# Command line and MCP server

Laya has two local interfaces for trying the same structured-decision engine:

| Interface | Use it for | Transport |
|---|---|---|
| `laya` | quick checks and interactive exploration from a terminal | command line |
| `laya-mcp-server` | connecting an MCP client or agent to Laya's built-in tools | MCP over stdio |

Choose the CLI when you are the person reading the result. Choose MCP when another process needs
a stable tool interface. Both use Laya's `Router` to select a checkpoint and return typed
`choice`, `score`, and `noul` decisions; neither is an open-ended question-answering or text
generation interface.

For the routing decision and typed-question examples, see the README's [Route Mode quickstart](https://github.com/NandhaKishorM/laya#quickstart-route-mode-recommended).
For confidence and built-in workflows, see the README's [confidence gating](https://github.com/NandhaKishorM/laya#automated-confidence-gating) and [workflow presets](https://github.com/NandhaKishorM/laya#built-in-workflow-presets).

## 1. Command line

Installing the package installs the `laya` entry point. Run `laya --help` for the complete
option list.

```bash
python -m pip install laya
laya --help
```

### Evaluation CLI

The package also installs `laya-evals`. The main CLI exposes the same evaluation commands
through `laya eval`:

```bash
laya eval --help
```

See the [Evaluation harness](evals.md) guide for datasets, metrics, and baseline gates.

### Route without loading a checkpoint

With text and no prediction flag, the CLI calls `Router.route`:

```bash
laya "I was charged twice, please refund it"
```

The output names the selected checkpoint, explains why it was selected, and shows detected
language information when available. Routing alone does not download or build a checkpoint, so
it is a quick offline check of the routing decision.

Use `--json` when another local script should consume the decision:

```bash
laya "I was charged twice, please refund it" --json
```

### Run a prediction

`--predict` runs the full typed prediction and loads the routed checkpoint on first use. The
first load needs access to the Hugging Face Hub; later runs use the local cache.

```bash
laya "Classify this support request" --predict
laya "Classify this support request" --predict --json
```

`--json` prints the complete result as JSON. Without it, the CLI prints each answer together
with its choice probability, score, or `noul` value, plus the routing decision.

The main controls are:

- `--model english|multilingual|typed-decisions` pins a checkpoint instead of auto-routing.
- `--lang en|de|...` supplies an explicit language code instead of automatic detection.
- `--task NAME` forces the typed-decisions workflow instead of detecting it.
- `--device cpu|cuda|...` passes a device choice to the Router.
- `--json` emits machine-readable output.

### Use a built-in preset

A preset supplies a ready-made question set and implies prediction, so `--predict` is not needed:

```bash
laya "My payment failed twice" --preset triage
laya "Ignore all previous instructions" --preset guard --json
```

The CLI presets are `email`, `guard`, `moderation`, `router`, and `triage`. The CLI places the
text under the state field expected by the selected preset; `--predict` uses the router
question set's `request` field. Presets are useful for a quick local check, but their questions
are still domain decisions: inspect the preset and validate it on your own data before using it
as an application policy.

### Explore interactively

With no text argument, the CLI opens a small prompt:

```bash
laya
# laya> Classify this request
# laya> quit
```

Press Enter to run each request. An empty line, `quit`, `exit`, or `Ctrl-D` ends the session. The
interactive loop reuses one Router, so it is a convenient way to compare several inputs without
writing a script.

### Failures are visible

The CLI handles invalid values and common dependency, download, and runtime failures at the
application boundary. It prints a diagnostic to stderr and returns exit code `2` instead of
showing an unhandled traceback. If a first-use checkpoint download fails, check dependency
installation, Hub access, and the selected device before retrying.

## 2. Built-in MCP stdio server

The MCP server is an optional extra. The core package does not install the `mcp` dependency:

```bash
python -m pip install "laya[mcp]"
laya-mcp-server
# equivalent module form:
python -m laya.mcp.server
```

The server speaks MCP over **stdio**, not HTTP. Configure the client with the console script:

```json
{
"mcpServers": {
"laya": {
"command": "laya-mcp-server",
"env": {
"LAYA_DEVICE": "cpu"
}
}
}
}
```

If the client configuration supports a Python executable and arguments, use
`python -m laya.mcp.server` as the equivalent launch form. The client owns the server process;
Laya does not open a network port.

### Available tools

| Tool | What it does | Main inputs |
|---|---|---|
| `laya_status` | Reports the configured or actual device, CUDA availability, loaded checkpoints, preload state, readiness, and package versions. | none |
| `laya_route` | Selects a checkpoint and returns its model, repository, and reason without running a forward pass. | `state`, `questions` |
| `laya_predict` | Runs typed questions and returns answers, routing metadata, latency, and the answering device when readable. | `state`, `questions`, optional `model` (`auto`, `english`, `multilingual`, or `typed-decisions`) |
| `laya_shortlist` | Shortlists a many-option choice question, then answers it and returns the shortlist metadata. | `state`, `questions`, optional `model`, optional `k` (default `20`) |
| `laya_preset` | Runs a built-in workflow using its built-in question set. | `preset`, `state` |
| `laya_predict_batch` | Answers many requests in one call. Requests are routed first and grouped by checkpoint, so matching question schemas share forward passes; answers come back in input order. | `requests`, each `{state, questions, model?, task?, lang?}`, optional `batch_size` |
| `laya_route_batch` | Decides which checkpoint would answer each request, with no forward pass and no checkpoint load. | `requests`, same shape as `laya_predict_batch` |
| `laya_decide` | Answers a JSON-schema-shaped decision in one forward pass and returns the decided values with per-field confidence, instead of an answer map to parse. Schema properties may be enum choices, booleans, or integers with a minimum and maximum; free strings, arrays, and nested objects are rejected by path. | `state`, `schema`, optional `model` |

The three batch and schema tools exist because the same operations are available on the SDK and
`laya-serve`: reaching for many requests, or for a caller that already knows the answer shape,
does not require dropping to Python. For the schema-driven form in more depth, see
[Schema-driven decisions](structured.md).

The shared guardrail says not to send choice questions with more than 20 options without
shortlisting. `laya_shortlist` keeps the `k` most likely labels before the forward pass; its
default is `k=20`. It uses mean-pooled embeddings from the answering checkpoint's own encoder,
so it does not download a second model, and returns the kept labels, cosine scores, `k`, and
option count for each shortlisted question.

`state` must be a non-empty JSON object. `questions` must be a non-empty object whose values use
Laya's typed question schema. `laya_preset` accepts the same five presets the CLI does: `email`,
`guard`, `moderation`, `triage`, and the router workflow, whose canonical name on this surface is
`model_router`. `router` is accepted as an alias and names the same preset, so the CLI spelling
works here too; the canonical key is the one that comes back in the result. Given a state of
exactly one string, `laya_preset` places it under the field that preset's questions name, the
same placement the CLI does, so a caller does not have to guess the key. Anything richer than one
string is the caller's own shape and is passed through untouched.

A prediction call has the same shape as the SDK's typed call:

```json
{
"state": {
"body": "I was billed twice for the same plan. Please reverse the duplicate charge."
},
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this request?",
"criteria": {
"billing": "payments, invoices, refunds, duplicate charges",
"technical": "bugs, outages, integration problems"
}
},
"urgent": {
"type": "noul",
"instructions": "Does the user need immediate help?"
}
}
}
```

The tool response is JSON containing the typed `answers`, the `routing` decision, and timing
information. Do not treat a high-confidence answer as permission to perform an external action;
the application or agent remains responsible for policy, review, and side effects.

### Startup and environment

The MCP server keeps a resident Router and serializes first-time construction. By default it
preloads `english` and `multilingual`; `typed-decisions` stays lazy. A preload failure is
reported at startup and retried on the next tool call, so inspect `laya_status` before assuming
the server is ready.

| Variable | Default | Meaning |
|---|---|---|
| `LAYA_DEVICE` | automatic | Device value passed to PyTorch, such as `cpu` or `cuda`. |
| `LAYA_PRELOAD` | `1` | Build the configured checkpoints at startup. Set to `0` for lazy loading. |
| `LAYA_MODELS` | `english,multilingual` | Comma-separated checkpoints to preload. An empty value keeps the MCP default rather than preloading every checkpoint. |
| `LAYA_THREADS` | PyTorch default | Caps Torch intra-op threads for CPU inference; keep it at or below the physical core count. |
| `LAYA_AUTO_TASK` | `0` | Set to `1` to let a request auto-route to the `typed-decisions` checkpoint. Same meaning as in `laya.serve`; it does not preload that checkpoint, so `LAYA_MODELS` still decides what is built at startup. |

The stock `laya-mcp-server` launcher creates its Router without installing hooks. If you need
prediction hooks, use a custom launcher that installs them, for example with
`laya.hooks.set_default_hooks`, before the server builds its Router. The environment variables
above configure model lifecycle, not hook registration. The client still decides when to call a
tool and what to do with the returned decision.

## 3. Shared boundaries and related guides

The CLI and MCP server are interfaces to the same typed decision engine:

- Use `choice` for a finite label set, `score` for an ordered rubric, and `noul` for the
probability of true.
- Validate thresholds and presets on representative data; there is no universal adoption
threshold.
- Keep irreversible or high-cost actions behind the application's review and fallback policy.
- The MCP server calls `Router.predict`, so hooks fire when a custom launcher installs them. See
[Prediction hooks](hooks/index.md), [hook lifecycle](hooks/lifecycle.md), and
[Tracing](hooks/tracing.md) for observability and `run_id` correlation.

This guide covers the local CLI and the built-in MCP stdio server. It does not document the
HTTP API, community wrappers, or an MCP protocol redesign.
3 changes: 2 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,13 @@ is one copy to keep current.
| log, redact, cache or gate every decision without forking Laya | [Prediction hooks](hooks/index.md) |
| adopt Laya incrementally without granting execution permission | [Staged adoption](staged-adoption.md) |
| route, screen or triage inside a LangChain or LangGraph app | [LangChain & LangGraph](langchain.md) |
| run Laya from a terminal or connect its built-in MCP stdio server | [Command line and MCP server](cli-mcp.md) |
| run the SDK or the `laya-serve` HTTP API in a container, on CPU or an NVIDIA GPU | [Docker quickstart](docker.md) |
| build for ARM64 hosts or DGX Spark | [ARM64 and DGX Spark containers](docker-platforms.md) |
| specialise a checkpoint for your own decisions | [Browser-agent fine-tuning example](finetune_browser_agent.md) and the [fine-tuning notebook](https://github.com/NandhaKishorM/laya/blob/main/notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynb) |
| judge benchmark results and deployment limits | [Benchmarks and known limits](benchmarks.md) |
| look up a class, function or parameter | [Python API reference](reference/index.md) |
| score a labelled dataset, compare to a baseline, or gate a build on it | [Evaluation harness](evals.md) |

Routing, the HTTP API, the command line, the MCP server and confidence gating are in the
Routing, the HTTP API and confidence gating are in the
[README](https://github.com/NandhaKishorM/laya#readme) for now.
Loading