Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -209,9 +209,9 @@ jobs:
-q

# The `test` job above deliberately stays free of `onnx`: this is the lane that
# installs it, so the exporter's quantization suite runs for real instead of
# skip-guarding into a trivial pass. Weight-free -- it builds tiny synthetic
# graphs, no checkpoint download.
# installs it, so the exporter's suites run for real instead of skip-guarding into a
# trivial pass. Weight-free -- it builds tiny synthetic graphs and a tiny stand-in
# model, no checkpoint download.
onnx-export:
name: onnx export (quantization, weight-free)
runs-on: ubuntu-latest
Expand All @@ -227,12 +227,19 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install -e .
pip install onnx onnxruntime
# The declared extra rather than a hand-listed pair: it also brings onnxscript, which
# torch.onnx.export needs for the torch.export path it takes by default.
pip install -e ".[onnx]"

- name: Quantized-export tests
run: python tests/test_onnx_quantize.py

# The only step that exports a graph and then runs it. tests/test_onnx_batch.py and
# tests/test_onnx_long.py drive ONNXAgent with a stub session, so neither can see an
# exporter that bakes the traced batch size into the graph -- which is what it did.
- name: Batch-dynamic export tests
run: python tests/test_onnx_export_batch.py

# The eval harness needs numpy and laya only, so it stays a separate, weight-free job: metric
# math, dataset parsing, the CLI and the API guard. The real-checkpoint gate is the scheduled
# `evals` workflow, which needs weights and network.
Expand Down
11 changes: 11 additions & 0 deletions docs/evals.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,17 @@ python scripts/export_onnx.py --model convaiinnovations/laya --output laya.onnx
laya-evals run data.jsonl --onnx laya.int8.onnx --max-ece 0.05
```

The export **verifies itself before it exits**: it runs the graph it just wrote in ONNX Runtime at
batch 1, 2 and 3 and compares against PyTorch, and `scripts/export_onnx.py --quantize` does the same
for the INT8 copy at a looser tolerance. A graph that cannot serve more than one question fails the
export instead of being written and deployed. That needs `onnxruntime` (`pip install laya[onnx]`);
`scripts/export_onnx.py --no-verify` skips it if you only have the exporter installed.

Marker width is dynamic in the exported graph for two or more markers (a graph traced at 2 runs at
3, 4, 5 and 8), but exactly one marker is baked out by a Python branch in the model, so a
one-criterion `score` question is not servable from ONNX when every question in the batch has one
criterion; use the torch path for those. There is no need to re-export per criteria count.

`--model` names the checkpoint the export came from — a Hub id or local path, not a Router
short name like `english`, since there is no Router on this path (default
`convaiinnovations/laya`). Its config and tokenizer are loaded from there. The agent serves one checkpoint, so a dataset row
Expand Down
Loading
Loading