Repository navigation
feat(rust): add Rust SDK (laya-onnx) - #672
Conversation
|
Holding this for the same maintainer decision as #671, not on the code. 8,627 lines of Rust with 44,172 lines of golden fixtures, all three checkpoints, both ONNX layouts, router, language detection, email and shortlist, running on ONNX Runtime with no Python at runtime. The approach is right and the verification strategy is the one I would ask for. The blocker is the same and it is mine: there is no CI lane that builds or tests The highest-value change is the same: a workflow that runs Leaving it open rather than closing. Thanks for the work, and for proposing it in #674 first rather than dropping it unannounced. |
|
Still holding, and now there is a template for getting it in. #671, the .NET SDK from the same author, merged for 0.3.26. The thing that moved it was the parity lane: That is the whole difference. This branch still has no So the ask is unchanged and now has a worked example to copy:
One thing I would do differently from #671 if you follow it: I merged that lane as advisory ( Also worth a rebase when you come back to it: The approach is right and the verification strategy is right. It is the maintenance seam that is still open. |
7282ede to
ef5bb7f
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Python float serialization is not byte-compatible as claimed, and the stated parity baseline and CI scope conflict with the implementation.
4 open findings
What changed in this PR
Adds the laya-onnx Rust SDK for local ONNX inference, with Python-parity tooling, fixtures, tests, samples, and advisory CI.
Changes:
- Implements core serialization, calibration, tokenization, configuration, email, and batching APIs.
- Adds golden-vector, property, routing, shortlist, and model-backed tests.
- Adds regeneration tools, sample crates, packaging metadata, and Rust CI.
| File | Description |
|---|---|
laya-rust/tools/test_ci_tools.py |
Tests CI utilities. |
laya-rust/tools/sample_cases.py |
Defines shared sample fixtures. |
laya-rust/tools/requirements-regen.txt |
Pins regeneration dependencies. |
laya-rust/tools/regen_golden.py |
Regenerates exports and goldens. |
laya-rust/tools/hashing_embedder.py |
Provides deterministic demo embeddings. |
laya-rust/tools/check_test_skips.py |
Rejects skipped or incomplete tests. |
laya-rust/tests/shortlist_end_to_end_tests.rs |
Tests shortlist inference parity. |
laya-rust/tests/router_end_to_end_tests.rs |
Tests router inference and concurrency. |
laya-rust/tests/python_json_tests.rs |
Tests Python-compatible serialization. |
laya-rust/tests/proptest_tests.rs |
Adds property-based checks. |
laya-rust/tests/laya_email_tests.rs |
Tests email cleaning parity. |
laya-rust/tests/hashing_embedder_tests.rs |
Tests embedder vectors. |
laya-rust/tests/golden/typed-decisions/meta.json |
Records typed-decisions metadata. |
laya-rust/tests/golden/typed-decisions/index.json |
Indexes typed-decisions cases. |
laya-rust/tests/golden/typed-decisions/case_single_option.json |
Adds single-option golden. |
laya-rust/tests/golden/typed-decisions/case_mask_injection.json |
Adds mask-injection golden. |
laya-rust/tests/golden/typed-decisions/case_list_state.json |
Adds list-state golden. |
laya-rust/tests/golden/typed-decisions/case_choice_criteria_list.json |
Adds list-criteria golden. |
laya-rust/tests/golden/routing/meta.json |
Records routing fixture metadata. |
laya-rust/tests/golden/multilingual/meta.json |
Records multilingual metadata. |
laya-rust/tests/golden/multilingual/index.json |
Indexes multilingual cases. |
laya-rust/tests/golden/multilingual/case_single_option.json |
Adds single-option golden. |
laya-rust/tests/golden/multilingual/case_mask_injection.json |
Adds mask-injection golden. |
laya-rust/tests/golden/multilingual/case_list_state.json |
Adds list-state golden. |
laya-rust/tests/golden/multilingual/case_choice_criteria_list.json |
Adds list-criteria golden. |
laya-rust/tests/golden/english/meta.json |
Records English metadata. |
laya-rust/tests/golden/english/index.json |
Indexes English cases. |
laya-rust/tests/golden/english/case_single_option.json |
Adds single-option golden. |
laya-rust/tests/golden/english/case_mask_injection.json |
Adds mask-injection golden. |
laya-rust/tests/golden/english/case_list_state.json |
Adds list-state golden. |
laya-rust/tests/golden/english/case_choice_criteria_list.json |
Adds list-criteria golden. |
laya-rust/tests/golden_questions_tests.rs |
Tests question rendering parity. |
laya-rust/tests/common/test_artifacts.rs |
Locates test model artifacts. |
laya-rust/tests/common/stub_tokenizer.rs |
Provides a deterministic tokenizer stub. |
laya-rust/tests/common/routing_golden.rs |
Loads routing fixtures. |
laya-rust/tests/common/result_compare.rs |
Centralizes golden result comparison. |
laya-rust/tests/common/mod.rs |
Exposes shared test helpers. |
laya-rust/tests/common/hashing_embedder.rs |
Ports the demo embedder to Rust. |
laya-rust/tests/common/engines.rs |
Loads test engines and layouts. |
laya-rust/tests/collator_tests.rs |
Tests batch collation. |
laya-rust/src/tokenization/mod.rs |
Exposes tokenizer APIs. |
laya-rust/src/tokenization/laya_tokenizer.rs |
Defines tokenizer contracts. |
laya-rust/src/python_json.rs |
Implements Python-style JSON formatting. |
laya-rust/src/lib.rs |
Defines the crate’s public surface. |
laya-rust/src/laya_options.rs |
Defines checkpoints and engine options. |
laya-rust/src/laya_email.rs |
Ports email utilities. |
laya-rust/src/laya_config.rs |
Parses model configuration. |
laya-rust/src/error.rs |
Defines crate-wide errors. |
laya-rust/src/collator.rs |
Builds padded ONNX batches. |
laya-rust/src/calibration.rs |
Implements probability calibration. |
laya-rust/samples/laya-sample/Cargo.toml |
Configures the quickstart sample. |
laya-rust/samples/laya-sample/.gitignore |
Ignores sample artifacts. |
laya-rust/samples/laya-routing/src/hashing_embedder.rs |
Adds routing demo embeddings. |
laya-rust/samples/laya-routing/Cargo.toml |
Configures the routing sample. |
laya-rust/samples/laya-benchmark/README.md |
Documents benchmarking. |
laya-rust/samples/laya-benchmark/Cargo.toml |
Configures the benchmark sample. |
laya-rust/samples/laya-benchmark/.gitignore |
Ignores benchmark artifacts. |
laya-rust/CHANGELOG.md |
Documents the initial release. |
laya-rust/Cargo.toml |
Configures the workspace and crate. |
laya-rust/.gitignore |
Ignores Rust build and model artifacts. |
.github/workflows/rust.yml |
Adds advisory Rust parity CI. |
🧠 Review effort: Balanced
Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.
|
Hi @NandhaKishorM, I pushed the parity lane you described in this comment, modelled on the one in #671. The branch is rebased onto current What was asked for
Drift found and ported (no tolerance loosened, recorded numeric answers unchanged): routing reasons, email cleaning, script counting, score legends, usage and option diagnostics, error messages, clipped option-span collisions, and float Verification
Not verified: GPU paths and optional features. Nothing in the workflow publishes, and |



Closes #674
What is this
A Rust SDK in
laya-rust/: a port of the Python SDK's inference path (tokenization, sequence building, calibration, answers) running the official checkpoints exported to ONNX, on ONNX Runtime. No Python process runs at runtime.What it does
choice/score/noulanswers, withconfidenceandanswer_confidence, for all three checkpoints (multilingual, english, typed-decisions). It supports both the single-file and the split ONNX layouts.laya-onnx0.1.0 on crates.io, versioned on its own. It's ready to publish, but I haven't published anything; that's yours to do.What it unlocks
Rust services can call Laya in-process, with no Python sidecar.
Verification
laya(Python) 0.4.0 and are checked in undertests/golden/. CI re-records them from the Python code at each commit withlaya-rust/tools/regen_golden.py(it usesdump_golden.pyanddump_routing_golden.py), so the checked-in copies are only a starting point. There are 17 cases per checkpoint, plus tokenizer and JSON probes and routing probes.skip:and the stub and math tiers still run.Notes for reviewers
laya-rust/plus one new workflow,.github/workflows/rust.yml. No PyPI or Python package changes. The workflow is advisory (continue-on-error), runs on every push and pull request with no path filter, re-records the goldens from Python, and fails the run if a model-backed test skips. It never publishes.langandemailwere brought in line with Python 0.4.0 where the regenerated goldens showed drift.min_confidence/ abstention,predict_longandlang_temperaturesfeatures aren't ported yet.tools/export_onnx.pythemselves, and one set of exports would serve every ONNX-based SDK.laya-rust-v*) and publishing to crates.io;dependency CVEscheck was already failing onmainbefore this PR. It's tracked in [bug]: dependency CVEs job in security.yml fails on main: 7 advisories in dependencies of optional extras #645 and unrelated to this change.