Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
docs(sdk): restore server guide anchor
  • Loading branch information
baninaveen committed Sep 25, 2026
commit e88aed01708a7448e99974ab9658f7b56ea0d1d0
17 changes: 10 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -445,11 +445,14 @@ The hint only decides *English or not*: a code whose primary subtag is `en`, `en

---

## Self-Hosting: HTTP Server
## Self-Hosting: HTTP Server (Jev-compatible)

`laya-serve` exposes the `Router` over its local `POST /v1/systemone` HTTP
endpoint. Responses contain `choice`/`score`/`noul` answers and an
`{input_tokens, output_tokens}` usage block.
`laya.serve` exposes the `Router` over HTTP on the same `POST /v1/systemone`
wire protocol as TypeSafe's hosted Jev API. Laya's answer payload is already
schema-identical to what Jev returns (`choice`/`score`/`noul` answers and a
`{input_tokens, output_tokens}` usage block), so an existing Jev client — e.g.
the [`hs-jev`](https://github.com/getmissionctrl/hs-jev) Haskell client — just
needs its `baseUrl` repointed; nothing else changes.

```bash
pip install "laya[serve]" # adds fastapi + uvicorn + python-multipart
Expand All @@ -472,11 +475,11 @@ send `Authorization: Bearer <key>`). A client's `model` field is honoured when i
names a Laya checkpoint (`english`/`multilingual`/`typed-decisions`), otherwise
the router auto-selects by script/language.

For HTTP clients:
Three things differ from Jev when you port a client:

* **Options per question.** A question's options share the checkpoint's option budget, `head_max_len` (192 tokens on `laya`, 256 on the other two). Once they overflow it, around 20 options with a short description each, every option is trimmed to fit, so long or similar labels can reach the model reading the same. Once they no longer fit the window at all, the request is rejected with 422. With short labels such as `Queue 042: Tickets routed to queue 42` that happens above 126 options on `laya` and 254 on the other two; the exact point moves with the length of the instructions and labels. For more candidates, narrow them first with `predict_shortlist` ([Honest limits](#honest-limits)).
* **Options per question.** A question's options share the checkpoint's option budget, `head_max_len` (192 tokens on `laya`, 256 on the other two), not Jev's cap of 255 options. Once they overflow it, around 20 options with a short description each, every option is trimmed to fit, so long or similar labels can reach the model reading the same ([Where Jev leads](#where-jev-leads)). Once they no longer fit the window at all, the request is rejected with 422. With short labels such as `Queue 042: Tickets routed to queue 42` that happens above 126 options on `laya` and 254 on the other two; the exact point moves with the length of the instructions and labels. For more candidates, narrow them first with `predict_shortlist` ([Honest limits](#honest-limits)).
* **Score levels.** Every level needs a description. A `null` level is rejected with 422 rather than scored and echoed back in `legend`.
* **`confidence`** on `choice` and `score` answers is 1 minus normalised entropy, a measure of how concentrated the distribution is. For one calibrated number on every question type, gate on `answer_confidence`, the probability of the reported answer.
* **`confidence`** on `choice` and `score` answers is 1 minus normalised entropy, a measure of how concentrated the distribution is, not Jev's `(n·p_max − 1)/(n − 1)`. A threshold carried over from Jev does not transfer. For one calibrated number on every question type, gate on `answer_confidence`, the probability of the reported answer.

### Nix / NixOS

Expand Down
2 changes: 1 addition & 1 deletion docs/typescript-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,5 +65,5 @@ Tiny random weights verify transport and numerical parity, not pretrained
quality or performance.

See the [SDK guide](../sdk/typescript/README.md) for setup, examples, and npm
publishing. The package is published as `laya-client`. Python release workflows
publishing. The package will be published under the `laya-client` name. Python release workflows
are unchanged.