feat(gepa): GEPA/GEPA-Flow Pareto optimizers + docs alignment (ax-llm#341)

* feat(optimizer): introduce GEPA reflective evolution with Pareto sampling

Adds a sample-efficient optimizer with reflective mutation, Pareto-based candidate selection, optional crossover, and multi-objective support. Includes typed options and examples, and improves progress logging of total rounds.

* feat(optimizer): add GEPA-Flow for multi-module reflective evolution

Implements a Flow-aware GEPA variant that selects modules round-robin and supports system-aware merge across candidates. Adds AxFlow helpers to expose/set node instructions and exports the optimizer.

* feat(optimizer): make GEPA and GEPA-Flow Pareto-only to align with the paper and simplify the API; refactor examples to remove basic variants and promote Pareto examples

* refactor(optimizer): remove legacy single-objective compile and rename GEPA-Pareto to GEPA in logs/metrics

* docs(optimize): clarify that GEPA/GEPA-Flow use compile for Pareto; MiPRO continues to use compilePareto

* chore(gepa-flow): use flow() factory and add OptimizationStart logging; align labels to GEPA-Flow

* feat(gepa,gepa-flow): adopt per-instance Pareto selection (Alg. 2)

Implement paper’s Algorithm 2 candidate selection:
- Track per-instance scalar scores on validation/Pareto set (S matrix)
- Sample parent from non-dominated set weighted by per-instance wins
- Compute scalar as mean of multi-objective metrics per instance
- GEPA-Flow also samples second parent for merge via Algorithm 2
- Persist S for each accepted candidate to drive subsequent sampling

Rationale: align behavior with the GEPA paper and improve exploration vs. archive crowding-distance selection; sets the stage for optional scalar acceptance gating in a follow-up.

* refactor(optimizer): centralize Pareto helpers for GEPA/GEPA-Flow

Extract shared multi-objective utilities to paretoUtils and replace inline duplicates in gepa.ts and gepaFlow.ts. No functional changes; simplifies maintenance and sets up for paper parity.

* feat(gepa,gepa-flow): add Merge strategy and guards; parametrize Pareto size

GEPA: enable periodic instruction merge with cap and progress reporting. GEPA-Flow: add merge caps, ancestry/desirability guards per Appendix F, and make Pareto set size configurable via args. Keeps acceptance via minibatch Pareto dominance.

* feat(gepa,gepa-flow): align with GEPA paper (splits, μf, σ-accept, guards)\n\nIntroduce explicit D_feedback/D_pareto splits to control rollout budget; plumb evaluator textual feedback μ_f into reflection; default to σ-based minibatch acceptance with configurable epsilon; add scalarizer/metric-key for per-instance S and Pareto selection; implement system-aware merge guards (ancestor/outperforms, desirability, tried merges).

* feat(gepa,gepa-flow): source-parity merges, acceptance, and adapter path

- Schedule merges via mergesDue/lastIterFoundNewProgram and skip reflective on merge attempts
- Dominator-based pair + ancestor selection with desirability filter and duplicate-merge guard
- Targeted subsample for merge acceptance (new_sum ≥ max(parent sums)); full eval on accept
- Stricter minibatch acceptance; when adapter provided, also require minibatch sum(child) > sum(parent)
- Parent selection via per-instance fronts; honor maxMetricCalls; preserve fallback behavior without adapter

This aligns both GEPA and GEPA-Flow with the reference engine while keeping public API stable.

* feat(gepa,gepa-flow): deterministic selection + strict acceptance for source parity

Seed RNG across selection/minibatching, enforce maxMetricCalls budget, and add Flow merge guards/de-dup for stable improvements; re-export adapter types and update examples.

* chore(cspell,gepa,gepa-flow): add GEPA terms; resolve lint warnings

Add domain terms (GEPA, Traj, etc.) to cspell and ignore dist to keep spelling checks green. Rename unused vars and drop unused imports in GEPA optimizers to satisfy lint without behavior changes.

* feat(gepa): source-parity single-module merges, guards, and safer defaults

Align GEPA merges with source: replace LLM merge with parent-pick, add ancestor/desirability guards and de-dup, use seeded sampling, schedule merges after accepted improvements; default merges off and skipPerfectScore on to match reference behavior.

* feat(gepa,gepa-flow,optimizer): epsilon ties, optional budget, aligned defaults

Bring GEPA in line with source parity by tolerating score ties, removing the hard budget requirement, and defaulting skip‑perfect in flow to match single‑module; Pareto frontier now respects epsilon.

* feat(gepa,gepa-flow): require maxMetricCalls for strict parity

Enforce a positive `options.maxMetricCalls` in GEPA/GEPA-Flow compile loops to match the source implementation and avoid unbounded optimization runs.

BREAKING CHANGE: compile now throws if `options.maxMetricCalls` is absent or non-positive.

* fix(gepa): only skip reflective after an evaluated merge attempt\n\nAlign single-module merge gating with the reference engine so reflective mutation is skipped only when a merge is actually attempted, improving behavioral parity and avoiding lost reflective iterations when no valid merge pair exists.

* docs(optimize): migrate multi-objective docs to GEPA/GEPA-Flow using compile (remove compilePareto)

---------

Co-authored-by: Spacy <832235+dosco@users.noreply.github.com>

Author: 2 people

.cspell/project-words.txt

@@ -144,5 +144,21 @@ anns
 annos
 mundo
 fmap
+GEPA
+gepa
+Traj
+vecs
+Scalarizer
+Scalarize
+scalarize
+scalarized
+idxs
+Instrs
+desirables
+subscores
+xorshift
+xorshift32
+Cand
+arrs
 PKCE
-MCPO
+MCPO

.gitignore

@@ -23,4 +23,6 @@ site/dist
 *.ac3
 *.webm
 *.txt
-__pycache__
+__pycache__
+
+gepa/

cspell.json

@@ -14,6 +14,7 @@
     "./src/examples",
     "./scripts",
     "./build",
-    "./dist"
+    "./dist",
+    "src/**/dist"
   ]
 }

docs/OPTIMIZE.md

@@ -750,13 +750,16 @@ const optimizer = new AxBootstrapFewShot({
 });
 ```
 
-### 2. Multi-Objective Optimization with `compilePareto`
+### 2. Multi-Objective Optimization with GEPA and GEPA-Flow
 
 **The Problem**: Sometimes you care about multiple things at once - accuracy AND
 speed AND cost. Traditional optimization only handles one objective at a time.
 
-**The Solution**: `compilePareto` finds the optimal trade-offs between multiple
-objectives using Pareto frontier analysis.
+**The Solution**: Use `AxGEPA` (single-module) or `AxGEPAFlow` (multi-module)
+with a multi-objective metric. Both use `compile(...)` and return a Pareto
+frontier of trade-offs plus hypervolume metrics.
+
+> Note: Pass `maxMetricCalls` in `compile` options to bound evaluation cost.
 
 #### What is Pareto Optimization?
 
@@ -774,7 +777,7 @@ Solutions A and B are both Pareto optimal (A is more accurate but
 slower/expensive, B is faster/cheaper but less accurate). Solution C is
 dominated by both A and B.
 
-#### When to Use `compilePareto`
+#### When to Use GEPA / GEPA-Flow
 
 ✅ **Perfect for:**
 
@@ -785,130 +788,96 @@ dominated by both A and B.
 
 ❌ **Skip for:**
 
-- Single clear objective (use regular `compile`)
+- Single clear objective (use regular `AxMiPRO.compile`)
 - When one objective is clearly most important
-- Quick prototyping (more complex than single-objective)
+- Quick prototyping (multi-objective adds complexity)
 
-#### Complete Working Example
+#### Complete Working Example (GEPA)
 
 ```typescript
-import { ai, ax, AxMiPRO } from "@ax-llm/ax";
+import { ai, ax, AxGEPA } from "@ax-llm/ax";
 
-// Content moderation with multiple objectives
-const contentModerator = ax(`
-  userPost:string "User-generated content" ->
-  isSafe:class "safe, unsafe" "Content safety",
-  confidence:number "Confidence 0-1",
-  reason:string "Explanation if unsafe"
+// Two-objective demo: accuracy (classification) + brevity (short rationale)
+const moderator = ax(`
+  userPost:string "User content" ->
+  isSafe:class "safe, unsafe" "Safety",
+  rationale:string "One concise sentence"
 `);
 
-// Training examples
-const examples = [
-  {
-    userPost: "Great weather today!",
-    isSafe: "safe",
-    confidence: 0.95,
-    reason: "",
-  },
-  {
-    userPost: "This product sucks and the company is terrible!",
-    isSafe: "unsafe",
-    confidence: 0.8,
-    reason: "Aggressive language",
-  },
-  // ... more examples
+const train = [
+  { userPost: "Great weather today!", isSafe: "safe" },
+  { userPost: "This product sucks and the company is terrible!", isSafe: "unsafe" },
+  // ...
 ];
 
-// Multi-objective metric function
-const multiMetric = ({ prediction, example }) => {
-  // Calculate multiple scores
-  const accuracy = prediction.isSafe === example.isSafe ? 1 : 0;
-
-  // Reward high confidence when correct, penalize when wrong
-  const confidenceScore = prediction.isSafe === example.isSafe
-    ? (prediction.confidence || 0)
-    : (1 - (prediction.confidence || 0));
-
-  // Reward explanations for unsafe content
-  const explanationScore = example.isSafe === "unsafe"
-    ? (prediction.reason && prediction.reason.length > 10 ? 1 : 0)
-    : 1; // No penalty for safe content
+const val = [
+  { userPost: "Reminder: submit timesheets", isSafe: "safe" },
+  { userPost: "Data breach follow-up actions required", isSafe: "unsafe" },
+  // ...
+];
 
-  // Return multiple objectives
-  return {
-    accuracy, // Correctness of safety classification
-    confidence: confidenceScore, // Quality of confidence calibration
-    explanation: explanationScore, // Quality of reasoning
-  };
+// Multi-objective metric
+const multiMetric = ({ prediction, example }: any) => {
+  const accuracy = prediction?.isSafe === example?.isSafe ? 1 : 0;
+  const rationale: string = typeof prediction?.rationale === 'string' ? prediction.rationale : '';
+  const len = rationale.length;
+  const brevity = len <= 30 ? 1 : len <= 60 ? 0.7 : len <= 100 ? 0.4 : 0.1;
+  return { accuracy, brevity } as Record<string, number>;
 };
 
-// Set up optimizer
-const optimizer = new AxMiPRO({
-  studentAI: ai({
-    name: "openai",
-    apiKey: process.env.OPENAI_APIKEY!,
-    config: { model: "gpt-4o-mini" },
-  }),
-  examples,
-  options: { verbose: true },
-});
+const student = ai({ name: 'openai', apiKey: process.env.OPENAI_APIKEY!, config: { model: 'gpt-4o-mini' } });
+const optimizer = new AxGEPA({ studentAI: student, numTrials: 16, minibatch: true, minibatchSize: 6, seed: 42, verbose: true });
 
-// Run multi-objective optimization
-console.log("🔄 Finding optimal trade-offs...");
-const result = await optimizer.compilePareto(
-  contentModerator,
-  examples,
-  multiMetric,
+console.log("🔄 Finding Pareto trade-offs...");
+const result = await optimizer.compile(
+  moderator as any,
+  train,
+  multiMetric as any,
+  {
+    validationExamples: val,
+    // Required to bound evaluation cost
+    maxMetricCalls: 200,
+    // Optional: provide a tie-break scalarizer for selection logic
+    // paretoMetricKey: 'accuracy',
+    // or
+    // paretoScalarize: (s) => 0.7*s.accuracy + 0.3*s.brevity,
+  } as any
 );
 
-console.log(`✅ Found ${result.paretoFrontSize} optimal solutions!`);
-console.log(`📊 Hypervolume: ${result.hypervolume?.toFixed(4) || "N/A"}`);
+console.log(`✅ Found ${result.paretoFrontSize} Pareto points`);
+console.log(`📊 Hypervolume (2D): ${result.hypervolume ?? 'N/A'}`);
 
-// Explore the Pareto frontier
-result.paretoFront.forEach((solution, index) => {
-  console.log(`\n🎯 Solution ${index + 1}:`);
-  console.log(`  Accuracy: ${(solution.scores.accuracy * 100).toFixed(1)}%`);
-  console.log(
-    `  Confidence: ${(solution.scores.confidence * 100).toFixed(1)}%`,
-  );
-  console.log(
-    `  Explanation: ${(solution.scores.explanation * 100).toFixed(1)}%`,
-  );
-  console.log(`  Strategy: ${solution.configuration.strategy}`);
-  console.log(`  Dominates: ${solution.dominatedSolutions} other solutions`);
+// Inspect a few points
+for (const [i, p] of [...result.paretoFront].entries()) {
+  if (i >= 3) break;
+  console.log(`  #${i+1}: acc=${(p.scores as any).accuracy?.toFixed(3)}, brev=${(p.scores as any).brevity?.toFixed(3)}, config=${JSON.stringify(p.configuration)}`);
+}
+
+// Choose a compromise by weighted sum (example)
+const weights = { accuracy: 0.7, brevity: 0.3 };
+const best = result.paretoFront.reduce((best, cur) => {
+  const s = weights.accuracy * ((cur.scores as any).accuracy ?? 0) + weights.brevity * ((cur.scores as any).brevity ?? 0);
+  const b = weights.accuracy * ((best.scores as any).accuracy ?? 0) + weights.brevity * ((best.scores as any).brevity ?? 0);
+  return s > b ? cur : best;
 });
+console.log(`🎯 Chosen config: ${JSON.stringify(best.configuration)}`);
 ```
 
-#### Choosing the Best Solution
+#### GEPA-Flow (Multi-Module)
 
 ```typescript
-// Option 1: Pick the solution that dominates the most others
-const mostDominant = result.paretoFront.reduce((best, current) =>
-  current.dominatedSolutions > best.dominatedSolutions ? current : best
-);
-
-// Option 2: Pick based on your priorities (weighted combination)
-const priorities = { accuracy: 0.6, confidence: 0.3, explanation: 0.1 };
-const bestWeighted = result.paretoFront.reduce((best, current) => {
-  const currentScore = Object.entries(current.scores)
-    .reduce((sum, [obj, score]) => sum + score * (priorities[obj] || 0), 0);
-  const bestScore = Object.entries(best.scores)
-    .reduce((sum, [obj, score]) => sum + score * (priorities[obj] || 0), 0);
-  return currentScore > bestScore ? current : best;
-});
-
-// Option 3: Interactive selection based on business requirements
-const businessOptimal = result.paretoFront.find((solution) =>
-  solution.scores.accuracy >= 0.85 && // Must be at least 85% accurate
-  solution.scores.confidence >= 0.7 && // Must be well-calibrated
-  solution.scores.explanation >= 0.8 // Must explain unsafe content well
-);
-
-// Apply the chosen solution
-if (businessOptimal?.demos) {
-  contentModerator.setDemos(businessOptimal.demos);
-  console.log("🎯 Applied business-optimal solution");
-}
+import { AxGEPAFlow, flow, ai } from "@ax-llm/ax";
+
+const pipeline = flow<{ emailText: string }>()
+  .n('classifier', 'emailText:string -> priority:class "high, normal, low"')
+  .n('rationale', 'emailText:string, priority:string -> rationale:string "One concise sentence"')
+  .e('classifier', (s) => ({ emailText: s.emailText }))
+  .e('rationale', (s) => ({ emailText: s.emailText, priority: s.classifierResult.priority }))
+  .m((s) => ({ priority: s.classifierResult.priority, rationale: s.rationaleResult.rationale }));
+
+const optimizer = new AxGEPAFlow({ studentAI: ai({ name: 'openai', apiKey: process.env.OPENAI_APIKEY!, config: { model: 'gpt-4o-mini' } }), numTrials: 16 });
+const result = await optimizer.compile(pipeline as any, train, multiMetric as any, { validationExamples: val, maxMetricCalls: 240 } as any);
+console.log(`Front size: ${result.paretoFrontSize}, Hypervolume: ${result.hypervolume}`);
 ```
 
 #### Advanced Multi-Objective Patterns
@@ -966,45 +935,36 @@ const multiMetric = ({ prediction, example }) => ({
 #### Understanding the Results
 
 ```typescript
-const result = await optimizer.compilePareto(program, multiMetric);
+const result = await optimizer.compile(program, examples, multiMetric, { maxMetricCalls: 200 } as any);
 
 // Key properties of AxParetoResult:
 console.log(`Pareto frontier size: ${result.paretoFrontSize}`);
-console.log(
-  `Total solutions generated: ${result.finalConfiguration?.numSolutions}`,
-);
-console.log(`Best single score: ${result.bestScore}`);
+console.log(`Best scalarized score on frontier: ${result.bestScore}`);
 console.log(`Hypervolume (2D only): ${result.hypervolume}`);
+console.log(`Total candidates evaluated: ${result.finalConfiguration?.candidates}`);
 
-// Each solution on the frontier contains:
+// Each frontier solution contains:
 result.paretoFront.forEach((solution) => {
-  solution.demos; // Optimized examples for this solution
   solution.scores; // Scores for each objective
-  solution.configuration; // How this solution was generated
-  solution.dominatedSolutions; // How many other solutions this beats
+  solution.configuration; // Candidate identifier for this solution
+  solution.dominatedSolutions; // How many others this point dominates
 });
 ```
 
 #### Performance Considerations
 
-- **Runtime**: `compilePareto` runs multiple single-objective optimizations, so
-  it takes 3-10x longer than regular `compile`
-- **Cost**: Uses more API calls due to multiple optimization runs
-- **Complexity**: Only use when you genuinely need multiple objectives
-- **Scalability**: Works best with 2-4 objectives; more objectives =
-  exponentially more solutions
+- **Runtime**: GEPA/GEPA-Flow perform reflective evolution with Pareto sampling; time scales with `numTrials`, validation size, and `maxMetricCalls`.
+- **Cost**: Bound evaluations with `maxMetricCalls`; consider minibatching.
+- **Scalability**: Works best with 2–4 objectives; hypervolume reporting is 2D.
+- **Determinism**: Provide `seed` for reproducibility; `tieEpsilon` resolves near-ties.
 
 #### Tips for Success
 
-1. **Start with 2-3 objectives**: More objectives make it harder to choose
-   solutions
-2. **Make objectives independent**: Avoid highly correlated objectives
-3. **Scale objectives similarly**: Ensure all objectives range 0-1 for fair
-   comparison
-4. **Use business constraints**: Filter the Pareto frontier by minimum
-   requirements
-5. **Validate solutions**: Test multiple Pareto-optimal solutions in practice
-
+1. **Start with 2-3 objectives**: More objectives make selection harder.
+2. **Scale objectives similarly (0–1)** for fair comparison.
+3. **Use `paretoMetricKey` or `paretoScalarize`** to guide selection/tie-breaks.
+4. **Validate chosen trade-offs** on a holdout set aligned to business constraints.
+5. **Keep validation small** to control cost; use `validationExamples` and `feedbackExamples` splits.
 ### 3. Chain Multiple Programs
 
 ```typescript

src/ax/dsp/common_types.ts

@@ -1,6 +1,7 @@
 import type { AxAIService, AxLoggerFunction } from '../ai/types.js';
 import type { AxOptimizerLoggerData } from './optimizerTypes.js';
 import type { AxFieldValue, AxResultPickerFunction } from './types.js';
+import type { AxGEPAAdapter } from './optimizers/gepaAdapter.js';
 
 export type AxExample = Record<string, AxFieldValue>;
 
@@ -173,4 +174,9 @@ export interface AxCompileOptions {
   overrideCheckpointLoad?: AxCheckpointLoadFn;
   overrideCheckpointInterval?: number;
   saveCheckpointOnComplete?: boolean;
+  // GEPA core options (adapter-based)
+  gepaAdapter?: AxGEPAAdapter<any, any, any>;
+  skipPerfectScore?: boolean;
+  perfectScore?: number;
+  maxMetricCalls?: number;
 }

src/ax/dsp/optimizerLogging.ts

@@ -71,8 +71,17 @@ export const axCreateDefaultOptimizerColorLogger = (
                 ? cl.red(` ↓${Math.abs(improvement).toFixed(3)}`)
                 : '';
 
+          const totalRounds =
+            typeof data.value.totalRounds === 'number' &&
+            data.value.totalRounds > 0
+              ? data.value.totalRounds
+              : typeof (config as any).totalRounds === 'number' &&
+                  (config as any).totalRounds > 0
+                ? (config as any).totalRounds
+                : 0;
+
           formattedMessage =
-            `${cl.yellow('● ')}${cl.whiteBright(`Round ${data.value.round}/${data.value.totalRounds}`)}` +
+            `${cl.yellow('● ')}${cl.whiteBright(`Round ${data.value.round}/${totalRounds}`)}` +
             (config.trialNumber !== undefined
               ? cl.gray(` [Trial #${config.trialNumber}]`)
               : '') +