Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -473,7 +473,7 @@ The hint only decides *English or not*: a code whose primary subtag is `en`, `en
wire protocol as TypeSafe's hosted Jev API. Laya's answer payload is already
schema-identical to what Jev returns (`choice`/`score`/`noul` answers and a
`{input_tokens, output_tokens}` usage block), so an existing Jev client — e.g.
the [`hs-jev`](https://github.com/getmissionctrl/hs-jev) Haskell client — just
the [`hs-jev`](https://github.com/getmissionctrl/hs-jev) Haskell client or the [`marcreichel/laya-php`](https://github.com/marcreichel/laya-php) PHP / Laravel SDK — just
needs its `baseUrl` repointed; nothing else changes.

```bash
Expand Down
2 changes: 1 addition & 1 deletion docs/http-api.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# HTTP API

`laya-serve` exposes Laya over the TypeSafe Jev `/v1/systemone` wire protocol. A client written
against Jev -- `hs-jev`, `typesafe-sdk`, or your own -- can point its base URL at this server and
against Jev -- `hs-jev`, `typesafe-sdk`, [`marcreichel/laya-php`](https://github.com/marcreichel/laya-php) (PHP / Laravel), or your own -- can point its base URL at this server and
keep working: Laya's `predict()` output is already schema-compatible, and the server adds only the
HTTP surface: one decision route, a health probe, an optional bearer check and request limits.

Expand Down
2 changes: 1 addition & 1 deletion laya/serve.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Laya's ``predict()`` output is already schema-compatible with the Jev decision
API -- ``choice`` / ``score`` / ``noul`` answers and a ``{input_tokens,
output_tokens}`` usage block -- so a client written against Jev (for example the
`hs-jev` Haskell client) can point its ``baseUrl`` at this server and keep
`hs-jev` Haskell client or `marcreichel/laya-php` PHP SDK) can point its ``baseUrl`` at this server and keep
working unchanged. All this module adds is the HTTP surface Laya itself does not
ship: a ``POST /v1/systemone`` route, an optional bearer check, and a health
probe.
Expand Down
10 changes: 5 additions & 5 deletions notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Fine-Tuning Laya on `LocalLLaMA/typed-decisions` (Kaggle GPU T4 \u00d72 DDP)\n",
"# Fine-Tuning Laya on `LocalLLaMA/typed-decisions` (Kaggle GPU T4 ×2 DDP)\n",
"\n",
"[![PyPI version](https://img.shields.io/pypi/v/laya.svg)](https://pypi.org/project/laya/)\n",
"[![Hugging Face Model](https://img.shields.io/badge/%F0%9F%A4%97%20Model-convaiinnovations%2Flaya-blue)](https://huggingface.co/convaiinnovations/laya)\n",
Expand All @@ -14,7 +14,7 @@
"\n",
"---\n",
"\n",
"### \u26a0\ufe0f Kaggle Notebook Settings:\n",
"### ⚠️ Kaggle Notebook Settings:\n",
"In the right-hand sidebar panel under **Notebook options**:\n",
"* **Accelerator:** Select **`GPU T4 x2`** (ensure both GPUs are allocated)\n",
"* **Internet:** Set to **`On`**\n",
Expand Down Expand Up @@ -171,7 +171,7 @@
"from torch.nn.parallel import DistributedDataParallel as DDP\n",
"from safetensors.torch import load_file, save_file\n",
"from transformers import AutoTokenizer\n",
"from laya.common import build_model, proper_reward, QTYPES\n",
"from laya.common import build_model, proper_reward, QTYPES, TEMP_MIN, TEMP_MAX\n",
"\n",
"def collate_train_batch(items, pad_id):\n",
" n, L = len(items), max(len(it[\"ids\"]) for it in items)\n",
Expand Down Expand Up @@ -215,7 +215,7 @@
" loss.backward()\n",
" return loss\n",
" opt.step(closure)\n",
" return float(torch.clamp(log_t.exp(), 0.1, 10.0).item())\n",
" return float(torch.clamp(log_t.exp(), TEMP_MIN, TEMP_MAX).item())\n",
"\n",
"def main():\n",
" dist.init_process_group(\"nccl\")\n",
Expand Down Expand Up @@ -431,7 +431,7 @@
" dist.destroy_process_group()\n",
"\n",
"if __name__ == \"__main__\":\n",
" main()\n"
" main()"
],
"execution_count": null,
"outputs": []
Expand Down
21 changes: 20 additions & 1 deletion tests/test_calibration_persistence.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
from transformers import BertConfig, BertModel, PreTrainedTokenizerFast # noqa: E402

from laya import load # noqa: E402
from laya.common import DecisionModel, QTYPES # noqa: E402
from laya.common import DecisionModel, QTYPES, TEMP_MIN, TEMP_MAX # noqa: E402


def export_notebook_config(cfg, fitted_temps, output_dir):
Expand Down Expand Up @@ -164,6 +164,25 @@ def test_missing_temperatures_keep_unit_defaults(self):
self.write_config()
self.assert_inference_temperatures({(t, 2): 1.0 for t in QTYPES})

def test_notebook_fit_one_temp_clamps_to_common_bounds(self):
notebook = Path(__file__).resolve().parents[1] / "notebooks" / (
"laya_finetune_typed_decisions_2xT4_kaggle.ipynb"
)
cells = json.loads(notebook.read_text(encoding="utf-8"))["cells"]
script, = ["".join(c["source"]) for c in cells
if "".join(c["source"]).startswith("%%writefile ")]
start = script.index("def fit_one_temp(sel):")
end = script.index("\ndef main():", start)
fn_code = ast.parse(textwrap.dedent(script[start:end]))
scope = {"torch": torch, "TEMP_MIN": TEMP_MIN, "TEMP_MAX": TEMP_MAX}
exec(compile(fn_code, str(notebook), "exec"), scope)
fit_one_temp = scope["fit_one_temp"]

high_sel = [([10.0, 0.0], [0.5, 0.5]) for _ in range(20)]
t_high = fit_one_temp(high_sel)
self.assertLessEqual(t_high, TEMP_MAX)
self.assertGreaterEqual(t_high, TEMP_MIN)


if __name__ == "__main__":
torch.set_num_threads(1)
Expand Down