Repository navigation
fix(evals): write redirected output as utf-8 - #1045
Conversation
Redirected stdout is encoded with the locale's codec by default (cp1252 on Windows), which cannot hold a non-Latin question id, tag or language, so `laya-evals validate data.jsonl > out.txt` died with a traceback and exit 1, the code kept for a quality failure. evals_cli.main() now reconfigures stdout to utf-8, as cli.main() does; `laya eval` returns before that line. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
A note on CI, since six checks are red and this PR only touches
|
|
Merged. The exit code is the part that makes this more than a cosmetic encoding fix, and you are right to lead with it. Your reproduction is the right one: a one-row dataset whose question id is Hindi, redirected to a file, before and after, with both the exit code and the file contents shown. Explaining why #893 did not already cover this is what made it quick to review: This lands in the same release as #1001, which stopped All 85 suites pass on the merged tree. |
35 pull requests from 8 contributors. * LAYA_EXTRA_MODELS registers extra checkpoints on laya-serve (#1047, #919) * fine-tuning prints its optimizer-update budget before a run (#1014, #963) * the Kaggle DDP notebook converges on the shared training primitives (#1017, #887) * evals writes redirected output as utf-8 (#1045) * the server serves again after a lifespan restart (#1049) * Java router lifecycle hooks, plus an OSV dependency scan (#1046) * twenty-four documentation corrections, each verified against the code Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
What
laya-evalsandlaya evalnow write stdout as utf-8, so output sent to a file or a pipe can hold a non-Latin question id, tag or language. This is the follow-up named at the end of #893, which did the same forlaya's own commands.Why
Redirected output is encoded with the locale's codec by default. On Windows that is cp1252 (Python before 3.15), which cannot encode most non-Latin text.
validateprints the dataset's question ids, andrun --sliceprints tags, languages and question ids, all of which come from the user's dataset. #893 did not cover this path:laya evalreturns fromcli.main()before the line that sets stdout, andlaya-evalshas its ownmain().Reproduced on Windows 11 with Python 3.12 and a one-row dataset whose question id is Hindi (
श्रेणी):laya eval validate ds.jsonl > out.txtfails the same way before and writes the same line after. The exit code matters here:docs/evals.mdkeeps1for a failed threshold or baseline tolerance, so a CI job on a Windows runner reads this crash as "the model got worse" when the dataset is valid.The fix is
sys.stdout.reconfigure(encoding="utf-8")at the top ofevals_cli.main(). It is guarded withhasattr, like the one incli.main(), because tests replace stdout.One behaviour change to know about, the same as in #893: on Windows, redirected output with accented Latin text (
é,ü) used to be written as cp1252 and is now utf-8.How it was verified
tests/test_evals.py: 138 passed. Onmainwithout this PR: 137 passed. With only theevals_cli.pychange removed, the new test fails with aUnicodeEncodeErrorat theprintin_cmd_validate.validatethrough both entry points with stdout wrapped in a latin-1 writer, so it fails on any OS without the fix, not only on Windows.tests/test_evals_shortlist.py: 7 passed.tests/test_evals_onnx.py: 33 passed, 0 failed.tests/test_evals_api.py,tests/test_cli.pyandtests/test_cli_lang_guess.py: all passed.Checklist
mainLeft out on purpose:
laya-evals run --slice tagend to end with a non-Latin tag, because that needs a checkpoint. It prints through the same stdout, so the same line covers it.laya-trainprints the output folder at the end (Training complete! Model saved to ...). A non-Latin folder name could hit the same problem when the output is redirected. I have not reproduced it, so it is not in this PR.