Repository navigation
Conversation
Independent calibration and evaluation need stable provenance and family boundaries. Preserve rendered schema order, reject conflicting duplicate labels, and expose one representative per group so repeated tickets cannot inflate statistical evidence.
Official test families must stay held out when derived variants or duplicate links are present. Record reservations in the verified manifest and keep zero-probability roles reserved.
Domain adaptation needs recoverable optimizer boundaries and exports that cannot inherit calibration fitted to different weights. Bind resume to the data, tokenized inputs, source assets, code and runtime, and reject silent input truncation.
Complete option markers and state do not prove the task instructions survived the head budget. Reject altered instructions before fitting or collecting evaluation evidence.
This was referenced Sep 22, 2026
This was referenced Sep 22, 2026
Owner
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add an opt-in local supervised adaptation runner that resumes the optimizer boundary after interruption and exports a standard SDK checkpoint. Restoring weights alone loses the optimizer, schedule, random states and data position; retaining a base checkpoint's temperatures after changing its weights also applies calibration fitted to a different model. The runner records those states, rejects changed inputs on resume, and exports explicitly unfitted calibration.
Changes
Testing
Additional validation on the minimum supported Python version: all 12 training tests pass on Windows with Python 3.10.11, CPU Torch 2.14.0, Transformers 5.17.0 and NumPy 2.2.6. The isolated test run used synthetic/local fixtures with Hub access disabled; it establishes compatibility, not model quality.
bbea5183a8f668ea14beaae92f81fa6fe39a2ea7c9f18bd7e0c127a1f58a6b57, source7b1d100) with the published renderer. The later instruction-truncation guard accepted every item (maximum 469 tokens, 77 options), and the training loop matches the frozen source. No model weights were loaded and no inference or accuracy measurement was performed.Notes for reviewer
This draft builds on #151; its data-contract commits are included as a dependency. The new runner is single-process fp32 supervised hard-label cross-entropy, not a replacement implementation of the notebook's RLCD objective. It complements #53's device-aware RLCD proposal and leaves #149's head activation checkpointing change separate. Calibration remains a subsequent held-out step, compatible with the direction of #19.
Exact resume across code/library/device changes, mixed precision and distributed training are deliberately outside this contract. Kernel locks and atomic replacement assume a local filesystem. No new dependency or GitHub workflow change is included.
Configuration correctness and reproducible execution are validated. An actual support-intent training/evaluation experiment is still running; no improvement in accuracy, calibration quality or automation coverage is claimed here.
Integration watch: #194 (reviewed at
4724305) changesAgent._to_internal(qdef)toAgent._to_internal(qid, qdef). A static AST/signature check confirms thatlaya/adapt_model.pycurrently supplies one argument, so combining those heads requires passing the question identity through the adaptation renderer and rerunning the affected training/evaluation/bundle tests. #194 is unmerged at upstreamc752770; this is an integration requirement, not a current-runtime failure or a claim that the combined changes pass. Active experiment sources remain frozen.