Skip to content

feat(adaptation): add resumable local supervised training - #159

Closed
alpha912 wants to merge 4 commits into
NandhaKishorM:mainfrom
MycelAI:alpha912/fix/resumable-adaptation
Closed

alpha912 wants to merge 4 commits into
NandhaKishorM:mainfrom
MycelAI:alpha912/fix/resumable-adaptation

Conversation

@alpha912

@alpha912 alpha912 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add an opt-in local supervised adaptation runner that resumes the optimizer boundary after interruption and exports a standard SDK checkpoint. Restoring weights alone loses the optimizer, schedule, random states and data position; retaining a base checkpoint's temperatures after changing its weights also applies calibration fitted to a different model. The runner records those states, rejects changed inputs on resume, and exports explicitly unfitted calibration.

Changes

  • Train hard-labelled choice, score and noul examples from the training partition only, using the existing sequence builder and DecisionModel. Default to a frozen encoder; support explicit encoder training with a separate learning rate. Keep the action head frozen.
  • Accumulate summed loss divided by the actual update's example count, including unequal final microbatches, and step the optimizer/scheduler once per update.
  • Persist model, optimizer, scheduler, Python/NumPy/Torch RNG and epoch/cursor state at atomic optimizer boundaries. Bind resume to data, tokenized inputs, source assets, implementation and runtime; use a kernel lock that releases on process death.
  • Load local assets without rewriting the source tokenizer. Reject truncated instructions, option descriptions, state and token-identical options before training.
  • Export neutral per-type temperatures, an empty bucket map and calibration-status provenance. Existing checkpoints and prediction APIs retain their intentional calibration precedence.

Testing

Additional validation on the minimum supported Python version: all 12 training tests pass on Windows with Python 3.10.11, CPU Torch 2.14.0, Transformers 5.17.0 and NumPy 2.2.6. The isolated test run used synthetic/local fixtures with Hub access disabled; it establishes compatibility, not model quality.

  • All 12 training regression tests pass on Windows and Linux, Python 3.12.11, CPU. Tiny local BERT/tokenizer fixtures require no downloaded weights.
  • Interrupted/resumed model, optimizer, scheduler, RNG, cursor and export files exactly match uninterrupted runs within each tested environment. Additional checks cover uneven accumulation, encoder freeze/update, source preservation, changed identities, failed writes, subprocess lock release, rendering loss and SDK reload probabilities.
  • Data contract (21), existing criteria (34), decision-model (3) and packaging (9) checks pass on both systems. Targeted Ruff E4/E9/F, compilation and whitespace checks pass.
  • Separately, a four-example training-only pilot used the actual pinned 322M multilingual checkpoint and all 77 Banking77 options on Linux CPU. Resumed/uninterrupted export files were identical and source assets unchanged. This establishes real-checkpoint execution/recovery, not model quality. The documented measured pilot used the earlier runner snapshot before the additional identity and instruction-truncation checks; final-source regression tests cover those additions.
  • A tokenizer-only audit of all 13,988 training items reproduced the frozen head run's complete input fingerprint (bbea5183a8f668ea14beaae92f81fa6fe39a2ea7c9f18bd7e0c127a1f58a6b57, source 7b1d100) with the published renderer. The later instruction-truncation guard accepted every item (maximum 469 tokens, 77 options), and the training loop matches the frozen source. No model weights were loaded and no inference or accuracy measurement was performed.
  • The actual head-only candidate completed all 438 optimizer updates over 13,988 training examples after verified CPU recovery from Windows to Linux. An independent audit matched the finalized checkpoint/export hashes and unchanged pinned base assets, and checked neutral temperatures in both JSON and the exported tensor with an empty bucket map. This verifies completed training and export integrity; calibration/policy evaluation is running and no final-test inference has occurred.

Notes for reviewer

This draft builds on #151; its data-contract commits are included as a dependency. The new runner is single-process fp32 supervised hard-label cross-entropy, not a replacement implementation of the notebook's RLCD objective. It complements #53's device-aware RLCD proposal and leaves #149's head activation checkpointing change separate. Calibration remains a subsequent held-out step, compatible with the direction of #19.

Exact resume across code/library/device changes, mixed precision and distributed training are deliberately outside this contract. Kernel locks and atomic replacement assume a local filesystem. No new dependency or GitHub workflow change is included.

Configuration correctness and reproducible execution are validated. An actual support-intent training/evaluation experiment is still running; no improvement in accuracy, calibration quality or automation coverage is claimed here.

Integration watch: #194 (reviewed at 4724305) changes Agent._to_internal(qdef) to Agent._to_internal(qid, qdef). A static AST/signature check confirms that laya/adapt_model.py currently supplies one argument, so combining those heads requires passing the question identity through the adaptation renderer and rerunning the affected training/evaluation/bundle tests. #194 is unmerged at upstream c752770; this is an integration requirement, not a current-runtime failure or a claim that the combined changes pass. Active experiment sources remain frozen.

Independent calibration and evaluation need stable provenance and family boundaries. Preserve rendered schema order, reject conflicting duplicate labels, and expose one representative per group so repeated tickets cannot inflate statistical evidence.
Official test families must stay held out when derived variants or duplicate links are present. Record reservations in the verified manifest and keep zero-probability roles reserved.
Domain adaptation needs recoverable optimizer boundaries and exports that cannot inherit calibration fitted to different weights. Bind resume to the data, tokenized inputs, source assets, code and runtime, and reject silent input truncation.
Complete option markers and state do not prove the task instructions survived the head budget. Reject altered instructions before fitting or collecting evaluation evidence.
@NandhaKishorM

Copy link
Copy Markdown
Owner

Thank you, resuming the optimizer, schedule and data position, not just the weights, is what makes a long run survivable. Since it is a draft and part of the larger series, I'll wait for you to mark it ready. Please rebase onto today's notebook changes (#191, #139) when you do.

@alpha912 alpha912 closed this Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants