Repository navigation
feat(train): add laya.train, one fine-tuning loop for the notebook and scripts to share - #899
Conversation
…d scripts to share The training loop lives in three drifting copies (the Kaggle notebook, the MPS script and research/scripts/finetune_single_device.py). This adds it to the package once, as the first step of NandhaKishorM#887. TrainConfig() trains the way the published notebook does: loss="rlcd", options in their listed order, and an effective batch of 64 (micro_batch=8 x grad_accum=8). - loss="soft-ce" trains on the soft cross-entropy alone (the variant NandhaKishorM#741 tested). - shuffle_options re-encodes items of the named types with a random option_order each epoch, through the same path inference uses, and permutes the target to match. Off by default. - Questions go through Agent._check_question and Agent._to_internal, so training sees the option text inference shows (list-form choices, custom noul labels, non-string instructions). - Calibration uses laya.calibrate.fit_temperature_map, so the fit follows the runtime clamp and buckets; an inherited temperature_by_options is dropped unless refitted. - Questions that cannot be trained on are skipped and counted by reason (invalid_question, invalid_target, target_mismatch, options_collapsed per NandhaKishorM#538). - The max_len / head_max_len used in training are written into the saved config, with the TrainConfig under training.laya_train. - freeze_encoder trains the head only and keeps the encoder in eval mode. The existing scripts are left untouched in this change; NandhaKishorM#851 and NandhaKishorM#885 are open against them. tests/test_train.py is weight-free (one-layer BERT, word-level tokenizer, no downloads) and is registered in the shared suite list and the Windows CI loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Workflows here are waiting for approval, so I ran the same CI on my fork: all green (CI 14/14 jobs, including the Windows and py3.10–3.13 test jobs, plus Docker, Docs and Security): https://github.com/GuilhermeFusari/laya/actions/runs/37129213237 |
|
Merged. I ran a real fine-tune with it rather than relying on the unit tests, because a training loop that passes weight-free tests and does not learn is the failure mode that matters here. It trainsA task the base checkpoint cannot do zero-shot: three options with arbitrary names (
The output directory loads with Option shuffling works, and this is the result I was most interested in#887's point 3 is that options are never shuffled during training and
So the augmentation does what the issue predicted, on real weights. That is a strong argument for the feature existing, and it is also why I am glad it defaults to off: it changes what the model learns, so it should be a decision rather than a surprise. On the designNot touching the three existing scripts was the right call given #851 and #885 were open against two of them, and it is why this merged in one pass. Routing the re-encode through Using One observation rather than a request: on my small run the fitted temperatures came back Verified
This is the first step of #887 and it is the one that needed to exist before the CLI and the CSV loader have anywhere to live. Thank you. |
From the review of #899: a short run can fit temperatures of [1.0, 1.0, 1.0] from a handful of calibration items and save a checkpoint whose confidences were never calibrated, with only the returned numbers to show it. calibration_report() describes, per question type, the calibration items the fit used (count, accuracy and mean confidence at the fitted temperature) and lists what to doubt: - fewer than MIN_TYPE_N items, so the temperature was not fitted and stays 1.0; - fewer than CALIB_WARN_N (50) items, so it was fitted on little evidence; - a temperature on the [TEMP_MIN, TEMP_MAX] clamp; - a fit that came back unchanged at 1.0, which usually means the items gave it nothing to correct. finetune() raises a RuntimeWarning for each issue, returns the report under "calibration" and saves it in the checkpoint config under training.laya_train_calibration. A type with no calibration items is reported without an issue, since it was not in the data. Nothing changes in what is trained or fitted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What
First step of #887. Adds
laya/train.py, a single training loop the notebook and the two scripts can later share. The existing scripts are not changed here, because #851 and #885 are open against them and I didn't want to conflict with either.TrainConfig: the notebook's training as the default (loss="rlcd", options in their listed order, effective batch 64 asmicro_batch=8, grad_accum=8, the same learning rates, cosine schedule, grad clip, sigma annealing and reward weights).loss="rlcd"(default) is the notebook objective: a GRPO term over noisy logits rewarded byproper_reward, plus soft cross-entropy.loss="soft-ce"trains on the soft cross-entropy alone, the variant Controlled A/B: GRPO + proper-reward term shows no measurable gain over (soft) cross-entropy in the fine-tune loop #741 tested.shuffle_options(for example("choice",)) are re-encoded with a randomoption_orderthroughbuild_sequence, the same path inference uses, and the target is permuted to match (target[option_order[s]]in slots). Items keep the question and the tokenized state, and states are tokenized once per row and shared by its questions.laya.calibrate.fit_temperature_mapon a slice held out before training, so the fine-tuned config uses the runtime clamp and buckets. An inheritedtemperature_by_optionsis removed unless the run refits it.max_len/head_max_lenused during training are written into the saved config, together with theTrainConfigundertraining.laya_train.finetune(data, model_dir, output_dir, config)runs the whole thing on a{state, questions, gold}JSONL and a local checkpoint directory, writescheckpoint_latest/after every epoch, and saves the final checkpoint in the layoutlaya.loadreads.No new dependency. Nothing is exported from
laya/__init__.py, so the public API contract intests/test_hooks_api.pyis unchanged.Why
The training loop lives in three copies that drift (#851 and #885 are the same temperature-clamp fix in two of them), and none of them is tested. With the loop in the package, the scripts can call one implementation, and the next PR in #887 can add a
laya-trainCLI and plain labelled-data input on top of it.How it was verified
Unit tests
tests/test_train.pyis weight-free: a one-layer BERT and a word-level tokenizer built in a temp directory, with downloads patched to fail. It covers:Agentrenders them (six shapes, including custom noullabels), so training sees the option text inference shows;build_sequence;soft_ce_lossagainst a hand computation, including padded options;rlcd_lossgiving the same value and gradient as the loss infinetune_single_device.pyunder the same RNG state;rlcd, no shuffling, effective batch 64);laya.loadopens and answers with.The suite is added to
scripts/test_suites.pyand to the Windows loop inci.yml.Results on typed-decisions (full fine-tune)
Run on Kaggle (2x T4, one run per GPU) with a notebook that installs
laya==0.3.24and adds this branch'slaya/train.py. Setup for every run:convaiinnovations/laya, the full train split (1,200 cases: 5,600 training items + 400 calibration items);max_len1024,head_max_len256, seed 0.Evaluation covers all 400 test cases (2,000 decisions), scored like cell 14 of the published notebook. "Reversed" means the same choice questions answered with the options in reverse order (
option_order), and "flip" is the share whose answer changes.layaloss="rlcd", no shufflingloss="soft-ce", no shufflingloss="soft-ce",shuffle_options=("choice",)laya-typed-decisionsPer workflow:
Paired differences on the same 2,000 decisions, with 95% bootstrap intervals over test cases:
What I take from it:
loss="rlcd"lands at 0.7715 against the published 0.766, so the module reproduces the notebook.rlcdas the default soTrainConfig()trains exactly like the notebook; switching the default, or retraining the published checkpoint withsoft-ce, is your call.BENCHMARKS.md), which I haven't measured.Smaller checks on my own machine (GTX 1650, 4 GB):
laya.load.freeze_encoder=True) on the GPU ran end to end.ampis a config option rather than always on for CUDA.Not in this PR
laya-trainand the CSV /expectedinput formats are the next PR.Checklist
mainlaya/__init__.py)Refs #887
🤖 Generated with Claude Code