Skip to content

fix(cli): report the median for even benchmark run counts - #2299

Open
MMexicann wants to merge 1 commit into
storytold:mainfrom
MMexicann:patch-30
Open

MMexicann wants to merge 1 commit into
storytold:mainfrom
MMexicann:patch-30

Conversation

@MMexicann

@MMexicann MMexicann commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

deckcraft-cli bench --runs 10 now reports the midpoint of the two middle timings as the median.

Follow-up to #1545. The benchmark picked the upper middle sample for even run counts, so the value labeled median was too high. Keep the existing sort and maximum calculation and use f64::midpoint for the median.

The regression checks the median and maximum of four unsorted measurements. A focused Rust probe reproduced the wrong median and passed four checks after the change. It also covered odd, empty, and equal large finite measurements. Full CLI tests were left to CI.

Summary by CodeRabbit

  • Bug Fixes
    • Corrected median reporting for even-sized timing samples.

@coderabbitai

coderabbitai Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

For even-sized samples, summarize_ms now reports the average of the two central sorted measurements. A timing test checks this median and the maximum.

Changes: Timing Summary

Layer / File(s) Summary
Median calculation and test
apps/deckcraft-cli/src/main.rs
summarize_ms averages the two central measurements for even-sized samples. The timing test checks a median of 11.0 and a maximum of 21.0 for four measurements.

Estimated code review effort: 2 (Simple) | ~5 minutes

Change: Bug fix

Suggested reviewers: harrison-booth

Merge Risk: Minimal · up to 8fa3c

The change makes the benchmark median for even run counts the midpoint of the two middle timings. It touches only timing summary output and is easily reversible. No merge-blocking risk remains.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: the CLI now reports the median for even benchmark run counts. The raw summary says summarize_ms now uses the midpoint of the two central sorted measurements for even-s…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: the CLI now reports the median for even benchmark run counts. The raw summary says summarize_ms now uses the midpoint of the two central sorted measurements for even-sized samples, which matches the title. The title follows the conventional commit format and is specific enough for a teammate scanning history.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 0ccc68f and 8fa3cca.

Files selected for processing: 1

  • apps/deckcraft-cli/src/main.rs

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 4 remain after this review.

@MMexicann
MMexicann marked this pull request as ready for review November 21, 2026 18:24
@autocraft-bot autocraft-bot Bot added the bugfix Fixes something that is broken: wrong result, crash, data loss label Nov 21, 2026

@autocraft-bot autocraft-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review (Claude Opus 5.5). Not a maintainer decision.

No blocking problems found (not certain).

For an even number of samples, summarize_ms now takes the median as the midpoint of the two middle sorted values. Odd and empty inputs behave as before.

  • apps/deckcraft-cli/src/main.rs:173-175: lower uses index (len-1)/2 and upper uses index len/2. For odd lengths both are the same element, so midpoint returns that element unchanged. For empty input both fall back to 0.0.
  • apps/deckcraft-cli/src/main.rs:765: the new assertion sorts [21, 4, 15, 7] to [4, 7, 15, 21] and expects median 11 and maximum 21, which matches the new logic.
  • f64::midpoint needs Rust 1.85 or later. If the workspace declares a lower rust-version, that value needs raising.

Claims checked:

  • verified: For even run counts the reported median is the midpoint of the two middle timings. (apps/deckcraft-cli/src/main.rs:173-175 takes ms[(len-1)/2] and ms[len/2] after sort_by(f64::total_cmp) and returns lower.midpoint(upper). The test at apps/deckcraft-cli/src/main.rs:765 expects 11.0 for [4,7,15,…
  • verified: The existing sort and maximum calculation are unchanged. (apps/deckcraft-cli/src/main.rs:172 still calls ms.sort_by(f64::total_cmp), and the maximum is still ms.last().copied().unwrap_or(0.0) at apps/deckcraft-cli/src/main.rs:175.)
  • verified: Odd and empty sample sets behave as before. (apps/deckcraft-cli/src/main.rs:173-174: for odd len, (len-1)/2 == len/2, so both values are the same element. For empty input both get() calls return None and fall back to 0.0. The test at apps/deckcraft-cli/src/main.rs:766 still expects (0.0, 0.0).)

Reviewed at 8fa3cca.

Recommendation: merge, after a maintainer's decision.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Fixes something that is broken: wrong result, crash, data loss

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant