🤖 Update Benchmark Results

[thomhurst]

Automated Benchmark Update

This PR updates the benchmark documentation with the latest results from the Speed Comparison workflow.

Benchmarks Produced

Individual benchmark artifacts are available for download:

• benchmark-DataDrivenTests
• benchmark-AsyncTests
• benchmark-ScaleTests
• benchmark-MatrixTests
• benchmark-MassiveParallelTests
• benchmark-SetupTeardownTests
• benchmark-BuildTime
• benchmark-summary (aggregated overview)

Runtime Benchmarks

• AsyncTests
• DataDrivenTests
• MassiveParallelTests
• MatrixTests
• ScaleTests
• SetupTeardownTests

Build Benchmarks

• BuildTime
• 2025-12-22
• Ubuntu Latest • .NET SDK 10.0.101

Changes

• Updated benchmark data in docs/static/benchmarks/latest.json
• Updated historical trends in docs/static/benchmarks/historical.json
• Regenerated benchmark documentation in docs/docs/benchmarks/index.md
• Updated benchmark summary in docs/static/benchmarks/summary.json

Workflow Run

• Run ID: 20418152813
• Triggered: schedule
• Date:

---

🤖 This PR was automatically created and will be merged automatically once CI checks pass.

[claude]

PR Review: Automated Benchmark Update

Summary

This is an automated PR that updates benchmark documentation with the latest CI results from 2025-12-22. The changes are systematic and appear to be generated correctly by the benchmark automation workflow.

---

✅ What's Good

1. Consistent Automation: All benchmark files are updated uniformly with the new date (2025-12-21 → 2025-12-22) and TUnit version (1.5.80 → 1.6.5)

2. Appropriate File Changes:

   • Updates individual benchmark markdown docs (7 files)
   • Updates JSON data files for each benchmark category
   • Updates historical tracking (historical.json)
   • Updates summary and aggregated data

3. Performance Trends Look Reasonable:

   • TUnit continues to maintain strong performance leadership across all benchmarks
   • AOT versions show expected significant performance gains
   • Variations in results are within normal statistical variance
   • No alarming performance regressions detected

4. Data Integrity: The timestamp consistency across all files (2025-12-22T00:31:09.537Z) indicates proper synchronization

---

📊 Notable Benchmark Changes (1.5.80 → 1.6.5)

TUnit Performance

• AsyncTests: 564.3ms → 585.9ms (+3.8%) - slight increase but within variance
• DataDrivenTests: 503.26ms → 505.32ms (+0.4%) - effectively stable
• MassiveParallelTests: 566.1ms → 614.6ms (+8.6%) - notable increase, worth monitoring
• MatrixTests: 573.44ms → 572.66ms (-0.1%) - stable
• ScaleTests: 517.94ms → 516.62ms (-0.3%) - stable
• SetupTeardownTests: 570.7ms → 571.7ms (+0.2%) - stable

Build Time

• TUnit: 2.148s → 2.067s (-3.8%) - small improvement ✅

---

⚠️ Observations

1. MassiveParallelTests Regression: The ~8.6% increase (566.1ms → 614.6ms) in MassiveParallelTests is the most notable change. While not alarming, it's worth investigating if this trend continues in future benchmarks.

2. AsyncTests Increase: The +3.8% increase in AsyncTests is also noticeable. Both of these could be:

   • Normal statistical variance
   • Environment/hardware differences in CI
   • Actual performance impact from changes in v1.6.5

3. AOT Performance Impact: TUnit_AOT also shows small increases:

   • AsyncTests: 122.3ms → 124.5ms (+1.8%)
   • MassiveParallelTests: 129.3ms → 131.9ms (+2.0%)

---

🔍 Recommendations

1. Monitor Next Run: Watch the next automated benchmark run to see if the MassiveParallelTests and AsyncTests increases persist or were just environmental variance

2. Consider Investigation: If the performance regression in parallel tests persists in subsequent runs, it may be worth investigating what changed between v1.5.80 and v1.6.5 that could affect parallel execution performance

3. No Blocking Issues: None of the changes warrant blocking this PR. The benchmark updates are legitimate and the performance variations are within reasonable bounds for different test runs

---

✅ Final Verdict

APPROVE - This is a clean automated benchmark update with no issues. The data appears consistent and valid. The slight performance variations observed are worth monitoring but don't indicate any critical problems.

The automation workflow appears to be functioning correctly, maintaining proper versioning, timestamps, and historical tracking.

---

Review completed following TUnit CLAUDE.md guidelines

[Copilot]

Pull request overview

This automated PR updates the TUnit benchmark documentation with the latest performance results from the December 22, 2025 Speed Comparison workflow run. The update includes refreshed performance metrics for TUnit version 1.6.5 compared against NUnit 4.4.0, MSTest 4.0.2, and xUnit v3 3.2.1 across multiple test scenarios.

Key Changes

• Updated TUnit version from 1.5.80 to 1.6.5 across all benchmarks
• Refreshed performance metrics for all benchmark categories (AsyncTests, DataDrivenTests, MassiveParallelTests, MatrixTests, ScaleTests, SetupTeardownTests, and BuildTime)
• Added new historical data entry for December 22, 2025

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
docs/static/benchmarks/summary.json	Updated timestamp to 2025-12-22
docs/static/benchmarks/latest.json	Updated timestamp, TUnit version to 1.6.5, and refreshed all benchmark metrics
docs/static/benchmarks/historical.json	Added new entry for 2025-12-22 benchmark run
docs/static/benchmarks/SetupTeardownTests.json	Updated timestamp, version, and performance metrics for setup/teardown tests
docs/static/benchmarks/ScaleTests.json	Updated timestamp, version, and performance metrics for scale tests
docs/static/benchmarks/MatrixTests.json	Updated timestamp, version, and performance metrics for matrix tests
docs/static/benchmarks/MassiveParallelTests.json	Updated timestamp, version, and performance metrics for massive parallel tests
docs/static/benchmarks/DataDrivenTests.json	Updated timestamp, version, and performance metrics for data-driven tests
docs/static/benchmarks/BuildTime.json	Updated timestamp, version, and build time performance metrics
docs/static/benchmarks/AsyncTests.json	Updated timestamp, version, and performance metrics for async tests
docs/docs/benchmarks/index.md	Updated date to 2025-12-22 and TUnit version to 1.6.5 in documentation
docs/docs/benchmarks/SetupTeardownTests.md	Updated date, version, performance table, and chart with latest metrics
docs/docs/benchmarks/ScaleTests.md	Updated date, version, performance table, and chart with latest metrics
docs/docs/benchmarks/MatrixTests.md	Updated date, version, performance table, and chart with latest metrics
docs/docs/benchmarks/MassiveParallelTests.md	Updated date, version, performance table, and chart with latest metrics
docs/docs/benchmarks/DataDrivenTests.md	Updated date, version, performance table, and chart with latest metrics
docs/docs/benchmarks/BuildTime.md	Updated date, version, build performance table, and chart with latest metrics
docs/docs/benchmarks/AsyncTests.md	Updated date, version, performance table, and chart with latest metrics