[OPIK-7038] [SDK] fix: deliberate tie policy + canonical no-improvement signal#7369
Merged
Merged
Conversation
…nt signal Optimization Studio sometimes surfaced the original (seed) prompt instead of the best-scoring trial: both optimizers only replaced the baseline when a candidate STRICTLY beat it, and the fallback was silent. - Add utils/scoring.improves_over as the shared keep-original-on-tie helper (strict >, None/NaN safe) and route the winner-vs-baseline / running-best comparisons through it across gepa, few-shot, evolutionary, hierarchical, meta_prompt and the base optimizer (behavior-preserving). - Expose a canonical details["reused_baseline"] flag from every result path (build_final_result, build_early_result, and the parameter optimizer's direct result), so "kept the original" is no longer silent or GEPA-only. - Reconcile the evolutionary winner against the baseline so a HOF/Pareto candidate that did not beat the seed no longer disagrees with the flag. Tie policy is unchanged (keep-original-on-tie) but now centralized, documented, and tested. Adds unit tests for the helper, GEPA selection, the base trial hook, the reused_baseline flag (final + early paths + metadata precedence), and the evolutionary reconciliation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
github-actions
Bot
added
python
Contributor
⏱️ pre-commit per-hook timing
⏭️ 25 skipped (no matching files changed)
|
![baz-reviewer[bot]](https://avatars.githubusercontent.com/in/933528?s=60&v=4){kind=small}
Comment on lines
+763
to
+764
| "reused_baseline": baseline_score is not None | ||
| and not improves_over(best_score, baseline_score), |
Contributor
There was a problem hiding this comment.
Reused-baseline mislabels tied parameters
select_best_trial() can mark details["reused_baseline"] from best_score vs. baseline_score, yet still copy a tied trial into details["optimized_parameters"] and details["optimized_model_kwargs"] when best_parameters is empty, so the no-improvement signal and returned payload disagree — should we fall back to base_model_kwargs whenever not improves_over(best_score, baseline_score)?
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
`sdks/opik_optimizer/src/opik_optimizer/algorithms/parameter_optimizer/parameter_optimizer.py`
around lines 763-764 inside `optimize_parameter` where the `details` dict is built
(specifically `reused_baseline`), the flag is derived from `best_score` vs
`baseline_score`, but `select_best_trial()` may still populate
`best_parameters`/`best_model_kwargs` from a tied non-baseline trial. Refactor by
computing `reused_baseline` first (using the existing `improves_over` check) and, when
`baseline_score is not None` and `not improves_over(best_score, baseline_score)`,
override `details[
Collaborator
There was a problem hiding this comment.
@awkoy, this is a valid concern; we should address it and make the trial selector strict as well, otherwise it can diverge with other logic.
Contributor
There was a problem hiding this comment.
Commit 11b0eb0 addressed this comment by making select_best_trial() strict on ties: it now only adopts a trial when it strictly improves over the incumbent, so tied trials no longer overwrite the baseline payload. The new tests also cover the no-improvement case to ensure optimized_parameters/optimized_model_kwargs stay aligned with reused_baseline.
Contributor
Author
There was a problem hiding this comment.
Good call — fixed in 11b0eb0 by making the selector itself strict rather than patching the details site.
select_best_trial() now routes the winner-vs-incumbent decision through the shared improves_over() helper and the old equal-score copy branch is gone:
if improves_over(best_value, best_score):
best_score = best_value
best_parameters = best_trial.user_attrs.get("parameters", {})
best_model_kwargs = best_trial.user_attrs.get("model_kwargs", {})
best_model = best_trial.user_attrs.get("model", best_model)So a tie deterministically keeps the incumbent (the baseline on the first call: empty parameters + baseline model kwargs). The selector is now the single source of truth for the tie policy, so reused_baseline and the returned optimized_parameters/optimized_model_kwargs can't disagree anymore. The strict-win path is unchanged (> ≡ improves_over for finite scores), and this was the only branch that could fire — after any real win best_parameters is non-empty, so the tie branch was reachable only in the pure no-improvement case.
Added test_select_best_trial.py covering strict-win / tie-keeps-baseline-no-leak / worse / empty-trials. Full parameter_optimizer + utils + core unit suites pass; ruff + mypy clean on the changed files.
… for mypy The tie-policy refactor dropped the explicit `context.current_best_score is not None` guard, relying on `improves_over` returning False for None. mypy can't infer that, so the `best_score = context.current_best_score` assignment tripped a float|None -> float error. Restore the explicit guard around `improves_over` (behaviorally identical) to satisfy the optimizer mypy lint. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
alexkuzmik
requested changes
Jul 9, 2026
alexkuzmik
left a comment
alexkuzmik
left a comment
Collaborator
There was a problem hiding this comment.
The changes look good! I left one comment only (reply to baz) about the diverging selection logic
Comment on lines
+763
to
+764
| "reused_baseline": baseline_score is not None | ||
| and not improves_over(best_score, baseline_score), |
Collaborator
There was a problem hiding this comment.
@awkoy, this is a valid concern; we should address it and make the trial selector strict as well, otherwise it can diverge with other logic.
… tie leak) Addresses PR review: select_best_trial() copied a tied (non-improving) trial's parameters/model_kwargs into the returned payload whenever the incumbent had empty parameters, while reused_baseline is derived from the strict improves_over() check — so on a tie the payload said "optimized" but the flag said "reused baseline". Route the selection through the shared improves_over() helper and drop the equal-score copy branch, so a tie deterministically keeps the incumbent (baseline on the first call). The winner path is unchanged (strict > == improves_over for finite scores). Adds unit coverage for the strict-win / tie / worse / empty-trials cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
alexkuzmik
approved these changes
Jul 13, 2026
awkoy
added a commit
that referenced
this pull request
Jul 14, 2026
…ine" on the overview Follow-up to the OPIK-7038 SDK fix (#7369), which made the tie policy deliberate (a candidate must STRICTLY beat the baseline; a tie keeps the seed) and added a canonical no-improvement signal. The UI previously crowned the max-score trial as "best" even when it did not beat the baseline, silently disagreeing with the SDK's optimized_prompt (which stays the seed). - useOptimizationExperiments: derive `improvedOverBaseline` = best.score > baseScore, mirroring the SDK's improves_over tie policy. - BestTrialPrompt: when there's no improvement, show a callout — "No improvement over baseline — the original prompt was kept." The best candidate then resolves to the baseline, so the panel shows the kept original prompt. Equals metric default (the other listed FE item) is already covered: the config form requires a non-empty reference key (schema.ts min(1)), so an Equals run can't ship with the empty default that caused flat scores. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
awkoy
added a commit
that referenced
this pull request
Jul 14, 2026
* [OPIK-7043] [FE] feat(optimization): run-config header pills + metric popover
Lift the single-run config (date, dataset, model, algorithm, metric) into the page header as pills, matching the revamped child-page headers. The metric pill opens a cursor-enterable HoverCard popover showing the metric's config. Reuses getOptimizerLabel / OPTIMIZATION_METRIC_OPTIONS / NavigationTag / HoverCard; adds a pure getOptimizationConfigItems resolver (+ tests) and a leaf MetricPopoverContent (+ tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] feat(optimization): dots grow on hover + pinned best-trial card
Chart now grows a dot on hover (getDotRadius) and shows the best trial's card by default, with the hovered dot's card overriding it. Extracts the card body into a reusable TrialCard (used by both the hover tooltip and the pinned default card); the default card is statically positioned and pointer-events-none so it never blocks dot hover. Adds getDotRadius + TrialCard tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] feat(optimization): Best trial prompt panel with baseline diff
Replace the overview's full-config dump with a prompt-focused 'Best trial prompt' panel that reuses the shared PromptComparison + buildPromptComparisonTargets (PR 1) for an inline baseline/parent diff, plus a 'View trial' link. Adds a pure getCandidatePrompt resolver (candidatePrompt.ts) + tests and a BestTrialPrompt component + render tests. Also tidies a prettier leftover in MetricPopoverContent.test.tsx.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(optimization): correct KPI duration + gate detail polling
KPI duration now measures to the run's completion time (last_updated_at) instead of the last trial's created_at, which under-reported by the last trial's runtime. Detail queries (optimization + trials) now stop their 5s refetch once the run finishes, via getOptimizationRefetchInterval, instead of polling completed runs forever. Both helpers extracted to optimizationOverviewHelpers.ts with tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] style(optimization): best-trial badge in card + prompt-section dot (Figma)
Match the Figma single-run spec: the trial card shows a 'Best trial' badge (and drops the redundant Status row) when it represents the best candidate — applied to both the pinned default card and the hover card on the best dot. Add the purple section dot before the 'Best trial prompt' heading.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] feat(optimization): surface run errors + stop swallowing log-fetch errors
On ERROR status the overview now shows a RunErrorPanel with the failure reason (extracted from the studio logs, since there is no structured error field) plus a 'View logs' action — instead of only the bare red status badge. useOptimizationStudioLogs no longer swallows every failure: a 404 (logs not written yet) stays graceful, but real fetch/decompress errors propagate, and OptimizationLogs shows a distinct error state. Adds extractErrorFromLogs (+ tests) and a RunErrorPanel render test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] style(optimization): align single-run overview with Figma
Verified against the Figma frame on a real seeded run. Header: larger title, back arrow, drop the 'Dataset:' pill prefix. Chart (optimization-feature): magenta dots/line, 'Baseline' first tick, 'Passed/Discarded trial' legend, purple 'Best trial' pill, rename Pruned->Discarded. KPI cards (optimization-feature): big current value + top-right trend badge + 'baseline -> best' sub-line. Diff view already reuses the new shared PromptComparison.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] style(optimization): 1:1 Figma polish — status dot, clock icon, fixed 0-1 y-axis
Header: status pill now carries a leading dot; date uses a clock icon (was calendar). Chart: pin the y-axis to a fixed 0-1 domain with quarter ticks (0.00-1.00) to match the Figma spec instead of a data-driven range.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] style(optimization): metric pill uses a chart icon (Figma)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(optimization): enlarge dot hover area, decouple from grow animation
Dots were hard to hover: the target was the small visible dot, and the grow-on-hover shifted that target (flicker). Add a fixed 18px transparent hit circle per dot (pointer-events: all) as the hover/click target, with the visible marks in a pointer-events:none group — so the hit area stays constant regardless of the dot's grow. Also make the decorative connector edges pointer-events:none so they no longer intercept dot hovers along the line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] feat(optimization): Figma-accurate run overview — StatCard KPIs, segmented tabs, header pills
- Add reusable StatCard compound component (ui/stat-card) and rebuild the KPI
cards on it, 1:1 with Figma (compact frame, colored trend pill, aligned values).
- PercentageTrend: opt-in compact "sm" size for the card badges (default unchanged).
- Switch Overview/Trials to the segmented-primary tabs variant.
- Header config/model/metric pills (OptimizationConfigPill, OptimizationModelPill).
- Progress chart + trial card refinements; claude integration icon.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] feat(optimization): Best trial prompt 1:1 + parse real optimizer prompts
- Best trial prompt panel matches Figma: white card, best-trial dot + 14px title,
"Diff vs baseline" + "View trial" header actions, plain-by-default with a diff toggle.
- Parse opik-optimizer's { "chat-prompt": [...] } wrapper as a single message array
(was falling through to a raw-JSON dump); covered by a unit test.
- PromptComparison gains showControls + controlled `diff` (backward compatible).
- Progress chart + TrialCard/TrialDetails refinements; move OptimizationConfigPill
into pages-shared/optimizations.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(optimization): show discarded trials as faded, unconnected dots
Progress chart now matches Figma for dataset runs, not just test suites:
- computeCandidateStatuses derives passed/pruned from the candidate tree for all
runs (was marking every dataset-run trial "passed", so nothing was discarded).
- ScatterDot paints pruned trials the lighter fuchsia-300 for all runs.
- buildParentChildEdges skips edges into pruned children, so the line follows
baseline → passed → best and discarded trials render as loose dots below it
(instead of the line diving down to every discarded child and back up).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: address PR review comments
- Only unwrap the opik-optimizer "chat-prompt" wrapper when it is the
object's sole key, so multi-prompt payloads containing a prompt named
"chat-prompt" keep their sibling prompts (classified as named prompts)
- Surface log-fetch failures in RunErrorPanel with a retry action
instead of silently hiding the logs entry point
- Move trial chart fuchsia shades (pruned dot, best dot, best ring) to
theme variables in main.scss with dark-mode values, and reuse them for
the BestTrialPrompt header dot
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: address follow-up PR review comments
* [OPIK-7043] [FE] chore: drop stale duplicate JSDoc on getUniqueSteps
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7044] [FE] feat(trial-sidebar): shell — panel, URL state, openers
Trial details now open in a right ResizableSidePanel over the run overview
(same shell + slide as the new-run form) instead of navigating away. URL
state keeps the old /trials route's `trials` + `trialNumber` param names so
deep links can redirect 1:1; open/close/switch are single batched replaceIn
updates, and closing clears the embedded table's params (trace/span/page/
filters/size/height). Header: info top bar with Trial #N, dataset tag, and
the optimization-logs button. Content lands in the next commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7044] [FE] feat(trial-sidebar): Results + Prompt tabs with status card
Sidebar body renders the trial-page content inside the panel: segmented
Results | Prompt tabs (Configuration is renamed Prompt per design), the three
metric KPI cards joined by a new Details status card (status dot from the
shared chart palette, Step tag, created date), and the test-items table.
The body reuses PageBodyScrollContainer (mx-0 h-full) so sticky headers and
the virtualized table work in the panel exactly as on a page. Status comes
from a single computeCandidateStatuses map lifted to OptimizationPage, and
parent resolution for the prompt diff is extracted into resolveTrialLineage.
All data comes from the run's already-loaded experiments — the sidebar
fetches nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7044] [FE] feat(trials-table): status dot, drop Trial items, hover diff button
Restyles the trials table per design: the Status tag gains a status dot
coloured from the shared chart palette and reads the page-level status map
(no more per-cell recomputation); the "Trial items" (trace_count) column is
removed; and the prompt cell's always-visible diff popover becomes a
row-hover GitCompare button ("View prompt diff" tooltip) that opens the
trial sidebar directly on the Prompt tab.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7044] [FE] refactor(trials): /trials route redirects into the sidebar; retire TrialPage
The /optimizations/$optimizationId/trials route now renders a small redirect
that forwards onto the run overview with the query params intact, so every
old deep link opens the same trial in the sidebar. The v2 TrialPage shell,
its TrialDetails header (whose pieces live in the panel top bar now), and the
long-dead ConfigurationTab/PromptTab are deleted; TrialKPICards and
TrialsItemsTab move under TrialSidebar/ next to their only consumer. v1 is
untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7044] [FE] refactor(table-types): type ColumnData.cell properly; drop as-never casts
ColumnData<T>.cell/.aggregatedCell were typed as tanstack's row-cell
*instance* type (Cell<T, unknown>) instead of its renderer template type —
the root cause of the `as never` cast on every cell assignment. They are now
ColumnDefTemplate<CellContext<T, unknown>>; existing casts elsewhere keep
compiling, and the optimization trial cells drop all 8 of theirs. The trial
cells are also typed CellContext<AggregatedCandidate, unknown> directly,
removing their per-cell `row.original as AggregatedCandidate` casts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7044] [FE] test(trial-sidebar): param state, status card, redirect, columns
Covers the sidebar URL-state hook (open/close/tab derivation, batched
open-trial resets, full close cleanup), the Details status card, the /trials
redirect (param forwarding + guard), and the trials columns (no trace_count,
status map + diff opener in meta, cast-free cell assignment).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7044] [FE] fix(trial-sidebar): diff button opens diff view; Best label on status card
Found in-browser: the prompt cell's diff button landed on the Prompt tab in
plain Configuration view. It now deep-links trialTab=diff, which opens
TrialConfigurationSection straight in Diff-vs-baseline (new optional
defaultViewMode prop, keyed remount per trial/view so the intent applies
while the sidebar stays open). The Details status card now reads "Best" for
the best trial, matching the trials table tag.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7043] [BE] feat: backend download route for studio optimization logs
Adds GET /studio/{id}/logs/download streaming the gzipped log file through
the backend. MinIO installations need it because the presigned URL can
point at a host the browser cannot reach (e.g. in-cluster MinIO); real S3
keeps using the presigned URL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(logs): fall back to backend download when presigned URL is unreachable
When fetching the presigned log URL fails for any reason other than 404
(in-cluster MinIO host, CORS), retry through the new backend
/logs/download route instead of surfacing a fetch error.
Also extracts the fullscreen scroll-ratio preservation into lib/scroll.ts
(with tests) and reuses it in the v1 and v2 log viewers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(chart): clamp the pinned best-trial card inside the chart container
Center the card under its dot but keep it within the container edges, so
a dot near the chart edge (e.g. a baseline-only run) cannot push the card
over the surrounding layout.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7044] [FE] feat: restyle the trials table to the Figma spec
Figma 689-34824 / 496-28178 (+ note 686-53223, diff popover 689-36080):
- Column set and order per design: Trial ("Trial #1" cells), Step, Prompt,
objective, "Opt. cost", Latency, Status, Created, with the spec widths
and right-aligned metric columns. "Trial items" and ID remain available
in the columns picker; "Trial items" leaves only the default selection.
- Metric cells render the compact PercentageTrend tag before the value.
- New shared TrialStatusPill (dot + label from the trial-status tokens;
the best trial gets the two-tone pale-fuchsia treatment), used by both
the table status cell and the trial details header.
- Prompt-cell diff button is hover-revealed with the previously missing
tooltip, and opens on hover or click; the popover matches the design
via a new opt-in "panel" variant of the shared PromptDiff (v1 call
sites keep the default look).
- Trials tab gains a search input, Figma-ordered controls (row size,
columns, refresh) and client-side pagination (50/page, page in URL).
- Fold-in refactor: type ColumnData.cell/header with ColumnDefTemplate
instead of the wrong tanstack instance types, dropping the "as never"
casts from the trial columns (one documented cast remains for the
shared TimeCell).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(chart): single trend line through the best trial per step
The progress chart forked the trend line whenever a step had multiple passed
trials (one edge per parent→child). Replace buildParentChildEdges with
buildTrendLineEdges, which connects the best-scoring baseline/passed candidate
of each step into one continuous path (base → step winner → …), matching Figma.
Non-winning passed, discarded, evaluating, and unscored trials render as loose
dots off the line; steps without a scored winner are bridged over.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7044] [FE] fix(trials): truly hide the prompt diff button until row hover, add separator
Figma 696:44233: the prompt cell shows a thin vertical separator before the
diff button, and both reveal together on row hover.
Switch the button from opacity-0 to the codebase's canonical
`hidden group-hover/row:flex` reveal (as IdCell uses) so it reserves no
width at rest — the prompt text now uses the full column until hovered,
matching the design. The separator + button stay visible while the diff
popover is open (driven by the existing open state).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7043] [FE] refactor(trials): virtualized flat table matching Figma
Match the Figma trials table: drop the Card wrapper (no shadow/border box) so
the table is flat with just a header underline and row separators, and switch
the body to DataTableVirtualBody with PageBodyStickyTableWrapper — the same
virtualized pattern the Experiments table uses.
This requires the page to provide a scroll container, so OptimizationPage is
wrapped in PageBodyScrollContainer and its header, tabs, overview content,
trials toolbar, and pagination move into PageBodyStickyContainer so they pin
correctly and keep page padding while the virtualized rows scroll under a
sticky column header.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(optimization-trials): diff-button icon style + configurable toolbar button sizes
- TrialPromptCell: replace bare icon with an outline Button (GitCompare) per Figma
- FiltersButton: add a `size` prop (defaults to sm)
- TrialItemsTab: use 2xs labeled toolbar buttons (filters, row-height, columns)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(trial-sidebar): align items-table toolbar with shared v2 tables
Match the flagship v2 data-table toolbar (LogsPage TracesSpansTab/ThreadsTab)
instead of the outdated Figma sizing: Filters stays icon, Row size uses the
shared icon-xs, and Columns uses the shared labeled "Columns N/N" at xs. Uses
the shared FiltersButton/DataTableRowHeightSelector/ColumnsButton as-is.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7044] [FE] chore: drop design-tool references from code comments
Comments should describe behavior and intent, not name the design tool or
its node IDs. Rewrites the affected comments across the optimization runs,
progress chart, prompt-diff and explain-plugin code, keeping the design
rationale but removing "Figma"/node-id metadata. Comment-only; no behavior
change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7044] [FE] refactor(trials): render the prompt diff trigger as an outline icon Button
Use the shared Button (variant outline, size icon-2xs) with the GitCompare
icon for the hover-revealed diff trigger instead of a bare styled button,
and drop the design-tool reference from its comment.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [OPIK-7043] [FE] refactor(trial-sidebar): reuse shared prompt-diff surface + match Figma picker
Prompt-diff picker (the "Compare against" select) now matches the Figma
"Version list" 1:1: a "Compare against" header, a separator, no radio
circles (selection shown via primary-100 bg + primary text), and each
option as title + optional muted "Trial #N" tag. Parent targets carry
their trial number in a new `caption` field, which also keeps multiple
crossover parents distinguishable and shows in the picker trigger.
Trial sidebar Prompt tab drops the bespoke TrialConfigurationSection /
resolveTrialLineage and reuses the shared PromptComparison surface via a
thin TrialPromptSection wrapper (baseline + parent targets resolved with
buildPromptComparisonTargets) — same experience as the overview
best-prompt panel and the trials prompt cell.
Also:
- TrialStatusCard: step tag restyled with a waypoints icon
- TrialItemsTab: section heading moved into the tab, toolbar aligned
- BestTrialPrompt: diff toggle swaps the git-compare / git-compare-off icon
- FiltersButton: accepts a className passthrough
- development plugin set gains a WorkspacePreloader re-export
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(trial-items): restore row virtualization, keep the bordered table box
The trial items table lost row virtualization; re-add DataTableVirtualBody
+ sticky header so large item sets virtualize like the sibling trials table.
Keep the full rounded border box while doing so: PageBodyStickyTableWrapper
now accepts an optional `className` (default "border-b", unchanged for
existing full-bleed callers) and the trial items table passes
"rounded-md border" via a small bordered wrapper.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(trial-items): drop the boxed border, full-bleed table
Manual forward revert of the bordered box added in the previous commit.
The trial items table should be full-width with a bottom border only
(matching Figma), so it goes back to the shared PageBodyStickyTableWrapper
default. Row virtualization + sticky header stay.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(trial-items): table sits inside a bordered container
Restore the rounded border box around the table (header + rows), with the
pagination below/outside the border, matching the Figma "Test items"
design. Forward-reverts the earlier border removal, which came from a
misread request. Virtualization + sticky header stay; text remains 14px
and the toolbar stays icon-only per product decision.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(trial-items): match the TraceLogsSidebar table exactly
Render the trial items table with the shared DataTable's default wrapper
(full-width, 1px rounded border, no virtualization/sticky header) — the
same setup TraceLogsSidebar uses — instead of my custom bordered sticky
wrapper (which used min-w-fit and so wasn't full width). Pagination stays
below/outside. PageBodyStickyTableWrapper is restored to its original,
untouched form (still used full-bleed by the sibling trials table).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(trial-items): tighten pagination gap under the table
Pagination container back to pb-4 (was py-4), removing the extra ~16px
of top padding that crept in during the table-wrapper churn. The table
area now matches the original spacing/padding rhythm (12px gaps).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: address PR review comments + post-merge variant fix
- useOptimizationTableState: clamp page against the current row count so a
stale/out-of-range page no longer slices an empty window when total shrinks.
- TrialsRedirect: translate legacy /trials params — page → itemsPage and
diff=true → trialTab=diff — so old deep links keep pagination and the diff view.
- Extract shared usePromptComparisonTargets hook; BestTrialPrompt and
TrialPromptSection now reuse it instead of duplicating target resolution.
- FiltersButton: restore variant={variant} dropped during the main merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] perf(trials-table): reuse the page-level status map per row
- TrialStatusCell recomputed computeCandidateStatuses(candidates) for every
row (O(n) per cell → O(n²) per render, re-run on every 5s poll while a run is
in progress). Thread the status map the page already computes through the
column meta so each cell is a single Map lookup.
- TrialMetricCells: drop the useBaselinePercentage useMemo whose cache never hit
(call sites pass fresh inline accessors); it's a plain helper now.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] remove studio log download BE endpoint + fallback
Drop the /studio/{id}/logs/download backend endpoint and its FE
fallback; the log viewer now fetches the presigned URL directly.
Includes prompt-diff surface refactor in shared CodeDiff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix(optimizations): dark card icons, Button-link CTA, 16px status tag
Design polish against Figma (Optimization Runs):
- Template cards: icon color white -> foreground-secondary #030712 (dark) so it
reads on the light chip; CTA ("Try template"/etc.) now uses the Button
variant=link size=2xs styling via buttonVariants (the card itself is a
<button>, so buttonVariants avoids invalid nested buttons).
- OptimizationStatusTag: fixed 16px pill with dot + label vertically centered
(was 20px, density-sized). Dropped the now-unused size prop and its
getCellTagSize/TAG_SIZE_MAP plumbing in OptimizationStatusCell.
tsc + eslint clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] feat(optimizations): trials chip filters, chart polish, test-suite pass cell
- Trials sidebar table: migrate filters to FilterChipBar/useFilterChips
(query-builder chips for dynamic columns + output + feedback scores, deduped
score options), title moved into the chip-bar prefix so chips wrap under it,
removed the client-side pass/fail toggle, 24px (2xs) row-size/columns controls.
- Reuse the experiment page's StatusTag + CellWrapper for the trial
passed/failed column instead of the bespoke Yes/No text cell.
- Optimization progress chart: foreground (#373D4D) axis ticks via fill,
2-decimal Y labels forced with interval={0}, #E2E8F0 grid, 20px grid rows,
raised top margin so the "Best trial" badge no longer clips, legend restyle.
- Optimization header + runs/trials toolbars: compact 2xs buttons.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: address Optimization runs review feedback
- show Item source column by default
- toolbar: on narrow screens filters wrap to their own full-width line
below the search; delete/columns/refresh stay in the top row
- prettify Algorithm & Metric values (map Python optimizer class names to
the studio enum + title-case fallback for unknown identifiers)
- Opt. cost cell: 14px -> 12px to match sibling metric columns
- feedback score pill uses the smaller "sm" variant (as in experiments)
- run error callout: smaller body text + inline underlined link retry
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] refactor: simplify optimizer label map + roll out Item source
- getOptimizerLabel: plain class-name -> label map instead of enum
normalization + regex prettifier; getMetricLabel reverted to the simple
options lookup
- bump only optimizations-selected-columns (v2 -> v3) so the default Item
source column reaches existing users, keeping width/order prefs intact
(matches how other tables version only the selected-columns key)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] fix: align trials tab with optimization table design QA
Address six design-team findings on the single-run Trials tab:
- Unify cell font size: drop hardcoded comet-body-s (14px) from Trial,
Step and Prompt cells so they inherit the row-height-driven size
(12px at small), matching the metric cells and restoring row-size
toggle responsiveness.
- Add min-w-0 truncate to Trial and Step cells so they ellipsize when
narrowed, like the other columns.
- HeaderWrapper now resolves alignment via resolveHorizontalAlignment,
honoring a column's horizontalAlignment override so a right-aligned
metric column keeps header and values on the same edge.
- Extract shared StatusDotPill primitive (list-table spec: h-5, px-2,
gap-1.5, centered content, truncating label). Route both
OptimizationStatusTag and TrialStatusPill through it, dropping the
trials pill's wrong padding/height/shrink-0. OptimizationConfigPill
(taller run-header config pills) is left untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] fix: trial sidebar design QA (callout, filters row, prompt diff)
Address four design-team findings on the trial sidebar:
- Remove the dismissable "what are trial items" explainer callout from
the Results tab (and its now-unused imports).
- Move the filter chips onto their own row below the section heading:
the "Test items"/"Evaluation results" heading and the Row size /
Columns controls stay on the top row, so a long filter set no longer
crowds the controls.
- Stop the prompt cards jumping when toggling the diff view: the plain
"Trial prompt" title now mirrors the target picker's box (border-b +
pb-px, transparent) so the control bar keeps a constant height across
the toggle.
- Baseline-align the "Parent" label and its "Trial #N" caption in the
compare-against picker (14px + 12px text) instead of centre-aligning
them to different heights.
The prompt-diff changes live in the shared PromptComparison surface, so
the overview best-prompt panel benefits too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE][SDK] fix: optimization overview design-feedback fixes
Addresses design-team feedback on the single-run Optimization overview:
- NavigationTag tooltip copy 'Navigate to' -> 'Go to' (item-source/dataset tag).
- Best-trial chart card: clamp vertically so a low-scoring best dot no longer pushes the pinned card past the chart onto the legend/links (mirrors the existing horizontal clamp).
- Prompt diff: LineDiff now delegates to the shared word-level TextDiff, so an unchanged leading sentence in an edited paragraph is no longer shown as both removed and added.
- Header algorithm & model pills now resolve for SDK-launched runs: FE falls back to metadata.optimizer/metadata.model, and the optimizer SDK records the model in optimization metadata (build_optimization_metadata).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: unify trials trend pills with optimization table
Address the remaining design-QA finding on the Trials tab: the trend
delta pills still used the filled green/red Tag (PercentageTrend size=sm)
while the optimization table used a neutral pill with a colored icon.
Extract the neutral pill into a shared MetricTrendPill primitive and route
both MetricComparisonCell (compact) and the trials metric cells through it,
so deltas render identically. Also tighten the trials cell gap 2 -> 1.5 to
match the optimization table's pill/value spacing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: wrap optimization-runs filters to their own row at lg
Design QA round 2: the filter chips only dropped below the search at the md
breakpoint (768px), so on smaller laptops / split views they still crowded the
search + actions row. Raise the wrap threshold md -> lg (1024px); search and the
right-aligned actions stay on the top row, filters wrap to a full-width block
below.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: show Algorithm and Metric columns by default
Design QA round 2: the pretty Algorithm/Metric labels landed but both columns
still shipped hidden. Add them to DEFAULT_SELECTED_COLUMNS. Bump the persisted
selection key v3 -> v4 so existing users pick up the new default instead of a
cached set that omits them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: inline the error-callout retry link
Design QA round 2: the retry was a block button on its own line below the
message. Render it inline at the end of the message paragraph as a text-sized,
always-underlined link (variant=link + underline, h-auto p-0, align-baseline).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: match metric popover to Figma spec
Design QA round 2 (Figma 686-53189): drop the redundant metric-name title (the
pill already names it), swap the parameter styles so the label is accented
foreground over a muted value, add a divider between parameter blocks, and
tighten the popover container padding to 6px (p-1.5, was the default 16px).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: best-trial popover reuses the standard chart tooltip
Design QA round 2: the best-trial card used bespoke <Customized> positioning
with hand-rolled left/top clamping, which parked it on top of the chart data.
Drop the manual positioning and reuse the shared ChartTooltip (a Radix popover
that flips/shifts to stay on screen), just opened by default for the best trial
while nothing else is hovered. ChartTooltip now takes bestCandidateId so the
card still shows the "Best trial" header styling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: resolve chart dot hover by nearest dot, not hit areas
Design QA round 2: hovering trial dots flickered and showed the wrong trial. The
per-dot invisible hit area (HIT_AREA_RADIUS 18 -> 36px target) was far larger
than the gap between clustered dots (~16px horizontally, ~14px vertically in a
fan-out), so overlapping hit circles fought over the pointer.
Make dots purely visual (pointer-events: none) and resolve hover + click with a
single nearest-dot handler on the chart container (findNearestDot, within
HOVER_HIT_DISTANCE). Proximity is unambiguous, so exactly one dot is ever active
- no thrash, and dense fan-outs are targetable. Extracted findNearestDot as a
pure, unit-tested helper. The in-progress ghost dot keeps its own handler.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: add gap between Parent label and trial caption
Design QA round 2: in the compare-against picker the "Parent" label and its
"Trial #N" caption sat flush together. Widen the baseline-aligned gap
gap-0.5 -> gap-1.5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: constrain chart trial popover to the chart + reposition on dot change
Follow-up to the hover/best-trial popover work:
- Constrain the popover to the chart container via collisionBoundary (Radix
otherwise only clamped to the portal root, so it overflowed onto the KPI
cards above the chart). It now flips/shifts to stay inside the chart.
- Key the popover by candidate id so it remounts when the target dot changes.
Radix only positions on mount, so moving quickly between dots updated the card
content but left it anchored to the previous dot; remounting repositions it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: don't remount best-trial popover when hovering the best dot
The best trial's popover is shown by default. Hovering the best dot promoted it
to the "hovered" trial, which swapped the default popover for an identical
hovered one and remounted it (re-animating the already-open card). Treat the
best dot as no hover so hoveredTrial stays null and the open popover stays put.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: optimization-runs filters on a dedicated bottom row
Replace the responsive single-row/wrap toolbar with a fixed two-row layout:
top row holds the search (left) and right-aligned actions (delete / columns /
refresh), and the filters sit on their own full-width row below. The previous
lg-inline layout crowded the filters into the middle of the top row on wide
screens.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] refactor: reuse the shared logs/traces filter-bar layout
Match the standard table toolbar used by logs/traces/threads: pass the search
as the FilterChipBar `prefix` so it shares one flex-wrap row with the filter
chips (they wrap to further lines as they grow, no odd cramming), and keep the
right-aligned actions (delete / columns / refresh) on the top row. The toolbar
prop type already omitted `prefix` for exactly this — we just weren't supplying
it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: single-row optimization-runs toolbar
Keep the search (FilterChipBar prefix) + filter chips and the right-aligned
actions (delete / columns / refresh) on the same row: the filter bar fills the
left as a flex-1 block that wraps to further lines as chips grow, and the
actions stay shrink-0 and top-aligned on the right.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [OPIK-7038] [FE] feat: surface "no improvement over baseline" on the overview
Follow-up to the OPIK-7038 SDK fix (#7369), which made the tie policy deliberate
(a candidate must STRICTLY beat the baseline; a tie keeps the seed) and added a
canonical no-improvement signal. The UI previously crowned the max-score trial
as "best" even when it did not beat the baseline, silently disagreeing with the
SDK's optimized_prompt (which stays the seed).
- useOptimizationExperiments: derive `improvedOverBaseline` = best.score >
baseScore, mirroring the SDK's improves_over tie policy.
- BestTrialPrompt: when there's no improvement, show a callout — "No improvement
over baseline — the original prompt was kept." The best candidate then
resolves to the baseline, so the panel shows the kept original prompt.
Equals metric default (the other listed FE item) is already covered: the config
form requires a non-empty reference key (schema.ts min(1)), so an Equals run
can't ship with the empty default that caused flat scores.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [OPIK-7038] [FE] fix: harden the no-improvement signal (code review)
Address findings from the review of the previous commit:
- Only assert "no improvement / original kept" on a COMPLETED run. Previously
the callout fired mid-run as soon as a scored trial hadn't yet beaten the
baseline, stating a final outcome for an unfinished run.
- Compare best vs baseline on the same aggregated-candidate path
(baselineCandidate.score) instead of the experiment-derived baseScore, which
defaults to 0 and is computed separately. This kills the dead `== null` guard
(baseScore was never null) and stops a divergent/absent baseline from faking a
comparison; an unscored baseline now reads as "not comparable".
- When there's no improvement, show the kept original (baseline) prompt as the
result via resultCandidate, so the panel no longer presents a losing trial's
prompt while the callout says the original was kept.
Not changed: the higher-is-better assumption (strict >) is intentional — it
mirrors the SDK's improves_over and the page-wide max=best convention.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: hide the always-open best-trial card while the trial sidebar is open
The best-trial popover floated over the open trial sidebar. Both render into the
same portal root, and the sidebar panel is an always-mounted z-10 layer, so no
single z-index sits both above the overview (card visible) and below the panel
(card hidden) — lowering it made the card vanish entirely. Instead skip
rendering the always-open card while the sidebar is open (suppressBestTrialCard,
driven by trialSidebar.open). Hover tooltips are unaffected — the chart is
covered while the sidebar is open, so there's nothing to hover.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] chore: remove unused OptimizationProgressTooltip
Dead code — the always-open <Tooltip open> component was not imported anywhere
(the chart uses ChartTooltip). Dropping it removes a stray instance of the
always-open-popover pattern.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: ignore stale dot positions in nearest-dot hit-testing
Review finding: ScatterDot writes into dotPositionsRef but never deletes, so a
candidate removed from a previous chartData could leave a stale position that
findNearestDot might return. Downstream guards already made a stale id a no-op,
but a stale entry could still block hovering a nearby live dot. Filter the ref
to candidates present in the current chart (candidateMap) before matching.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [OPIK-7043] [FE] fix: address review nits (empty prompt fallback, cache-safe column sort)
- getCandidatePrompt: skip an empty-string prompt so the section shows the
"No prompt available." fallback instead of a blank card (empty string yielded
no diff rows).
- TrialItemsTab: clone the React Query columns arrays with .slice() before
.sort(), so sorting no longer mutates the cached response in place.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details
Optimization Studio sometimes surfaced the original (seed) prompt instead of the best-scoring trial: both optimizers only replaced the baseline when a candidate strictly beat it, so a tie was discarded in favour of the original, and the fallback was silent. This PR keeps the tie policy (keep-original-on-tie) but makes it deliberate, centralized, and tested, and adds a consistent no-improvement signal. SDK-only; the FE/backend surfacing is a follow-up. Pairs with #7364 (OPIK_7160), which makes silent metric misconfig fail loudly and should land first.
utils/scoring.improves_over(score, baseline)— single tie-policy helper (strict>,None/NaN-safe); routed the winner-vs-baseline / running-best comparisons through it across GEPA, few-shot Bayesian, evolutionary, hierarchical-reflective, meta-prompt, and the base optimizer (behavior-preserving).details["reused_baseline"]flag set on every result path —build_final_result,build_early_result(perfect-score early-stop), and the parameter optimizer's direct result — so "kept the original" is no longer silent or GEPA-only. GEPA's legacyfinal_evaluation_reused_baselinekey is retained for back-compat.reconcile_winner_with_baselinereturns the seed + baseline score when the HOF/Pareto winner did not beat the baseline, so the returned prompt and the flag can't disagree.Change checklist
Issues
AI-WATERMARK
AI-WATERMARK: yes
reused_baselinesignal.Testing
sdks/opik_optimizereditable install in its.venv.python -m pytest tests/unit/algorithms tests/unit/core tests/unit/utils→ 696 passed (+4 new).test_scoring.py(strict/tie/worse/None/epsilon), GEPAtest_candidate_selection_baseline.py(the ticket's deterministic repro table), base_handle_trial_endkeeps incumbent on a tie,reused_baselineon the final + early/perfect-score paths + precedence over per-optimizer metadata, evolutionaryreconcile_winner_with_baseline.ruff checkandruff format --checkclean on changed files;mypyclean on changed files (one pre-existing unrelated error inparameter_optimizerbaseline eval).Documentation
N/A — SDK-internal result signal in
details; no public documentation affected. Surfacing it in the UI is a tracked follow-up.