Skip to content

Conversation

philippnormann
Copy link
Contributor

@philippnormann philippnormann commented Aug 8, 2025

Summary

Add docs showing how to initialize RL from an existing SFT LoRA by passing the adapter directory as the base_model when constructing art.TrainableModel. Includes a minimal example and concise motivation.

Changes

  • docs/fundamentals/art-client.mdx: add “Initializing from an existing SFT LoRA” section with TrainableModel example and “Why this?” (warm-start, small-model stability).
  • docs/getting-started/faq.mdx: add FAQ entry with short TrainableModel snippet.

Motivation

Many users fine-tune with SFT (e.g., Unsloth/PEFT) and want to continue with RL; pointing base_model on TrainableModel to the adapter directory is the simplest path and improves early training, especially for small models.

@bradhilton bradhilton requested a review from arcticfly August 8, 2025 12:34
@bradhilton
Copy link
Collaborator

LGTM, any feedback @arcticfly?

@arcticfly
Copy link
Contributor

Looks good! Thanks :)

@arcticfly arcticfly merged commit 75c761a into OpenPipe:main Aug 11, 2025
1 check passed
@JonesAndrew JonesAndrew mentioned this pull request Aug 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants