Skip to content

Commit f8663e0

Browse files
thomhurstthomhurst
authored
chore: update benchmark results (#3822)
1 parent d27cf3b commit f8663e0

18 files changed

+349
-345
lines changed

‎docs/docs/benchmarks/AsyncTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 2
77
# AsyncTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-12** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 558.9 ms | 557.2 ms | 4.11 ms |
20-
| NUnit | 4.4.0 | 650.2 ms | 649.0 ms | 9.00 ms |
21-
| MSTest | 4.0.2 | 615.0 ms | 614.8 ms | 5.89 ms |
22-
| xUnit3 | 3.2.0 | 702.4 ms | 702.4 ms | 7.09 ms |
23-
| **TUnit (AOT)** | 1.1.0 | 123.8 ms | 123.9 ms | 0.46 ms |
19+
| **TUnit** | 1.1.0 | 551.3 ms | 551.1 ms | 1.86 ms |
20+
| NUnit | 4.4.0 | 639.4 ms | 634.6 ms | 10.77 ms |
21+
| MSTest | 4.0.2 | 612.9 ms | 611.2 ms | 9.13 ms |
22+
| xUnit3 | 3.2.0 | 692.8 ms | 692.6 ms | 5.81 ms |
23+
| **TUnit (AOT)** | 1.1.0 | 122.7 ms | 122.8 ms | 0.41 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
5858
xychart-beta
5959
title "AsyncTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 843
62-
bar [558.9, 650.2, 615, 702.4, 123.8]
61+
y-axis "Time (ms)" 0 --> 832
62+
bar [551.3, 639.4, 612.9, 692.8, 122.7]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-12T21:13:20.516Z*
75+
*Last generated: 2025-11-13T00:28:35.380Z*

‎docs/docs/benchmarks/BuildTime.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 8
77
# Build Performance Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-12** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -18,10 +18,10 @@ Compilation time comparison across frameworks:
1818

1919
| Framework | Version | Mean | Median | StdDev |
2020
|-----------|---------|------|--------|--------|
21-
| **TUnit** | 1.1.0 | 2.033 s | 2.032 s | 0.0296 s |
22-
| Build_NUnit | 4.4.0 | 1.622 s | 1.615 s | 0.0216 s |
23-
| Build_MSTest | 4.0.2 | 1.705 s | 1.701 s | 0.0231 s |
24-
| Build_xUnit3 | 3.2.0 | 1.611 s | 1.613 s | 0.0248 s |
21+
| **TUnit** | 1.1.0 | 2.100 s | 2.101 s | 0.0207 s |
22+
| Build_NUnit | 4.4.0 | 1.688 s | 1.689 s | 0.0214 s |
23+
| Build_MSTest | 4.0.2 | 1.731 s | 1.733 s | 0.0235 s |
24+
| Build_xUnit3 | 3.2.0 | 1.648 s | 1.639 s | 0.0283 s |
2525

2626
## 📈 Visual Comparison
2727

@@ -60,7 +60,7 @@ xychart-beta
6060
title "Build Time Comparison"
6161
x-axis ["Build_TUnit", "Build_NUnit", "Build_MSTest", "Build_xUnit3"]
6262
y-axis "Time (s)" 0 --> 3
63-
bar [2.033, 1.622, 1.705, 1.611]
63+
bar [2.1, 1.688, 1.731, 1.648]
6464
```
6565

6666
---
@@ -69,4 +69,4 @@ xychart-beta
6969
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7070
:::
7171

72-
*Last generated: 2025-11-12T21:13:20.518Z*
72+
*Last generated: 2025-11-13T00:28:35.383Z*

‎docs/docs/benchmarks/DataDrivenTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 3
77
# DataDrivenTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-12** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 510.37 ms | 511.05 ms | 7.125 ms |
20-
| NUnit | 4.4.0 | 583.06 ms | 583.47 ms | 17.581 ms |
21-
| MSTest | 4.0.2 | 594.23 ms | 591.72 ms | 18.226 ms |
22-
| xUnit3 | 3.2.0 | 604.34 ms | 605.85 ms | 7.447 ms |
23-
| **TUnit (AOT)** | 1.1.0 | 25.55 ms | 25.62 ms | 0.241 ms |
19+
| **TUnit** | 1.1.0 | 486.77 ms | 486.53 ms | 3.791 ms |
20+
| NUnit | 4.4.0 | 589.92 ms | 591.78 ms | 12.153 ms |
21+
| MSTest | 4.0.2 | 605.63 ms | 602.89 ms | 14.207 ms |
22+
| xUnit3 | 3.2.0 | 600.41 ms | 597.06 ms | 10.504 ms |
23+
| **TUnit (AOT)** | 1.1.0 | 24.05 ms | 24.05 ms | 0.239 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
5858
xychart-beta
5959
title "DataDrivenTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 726
62-
bar [510.37, 583.06, 594.23, 604.34, 25.55]
61+
y-axis "Time (ms)" 0 --> 727
62+
bar [486.77, 589.92, 605.63, 600.41, 24.05]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-12T21:13:20.516Z*
75+
*Last generated: 2025-11-13T00:28:35.381Z*

‎docs/docs/benchmarks/MassiveParallelTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 4
77
# MassiveParallelTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-12** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 625.5 ms | 626.8 ms | 7.61 ms |
20-
| NUnit | 4.4.0 | 1,213.0 ms | 1,211.9 ms | 12.39 ms |
21-
| MSTest | 4.0.2 | 3,010.8 ms | 3,010.4 ms | 12.63 ms |
22-
| xUnit3 | 3.2.0 | 3,099.6 ms | 3,101.6 ms | 15.59 ms |
23-
| **TUnit (AOT)** | 1.1.0 | 132.3 ms | 132.3 ms | 0.36 ms |
19+
| **TUnit** | 1.1.0 | 601.9 ms | 602.5 ms | 2.97 ms |
20+
| NUnit | 4.4.0 | 1,181.6 ms | 1,177.9 ms | 13.31 ms |
21+
| MSTest | 4.0.2 | 2,965.3 ms | 2,966.7 ms | 6.96 ms |
22+
| xUnit3 | 3.2.0 | 3,066.6 ms | 3,066.7 ms | 6.78 ms |
23+
| **TUnit (AOT)** | 1.1.0 | 131.6 ms | 131.7 ms | 0.37 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
5858
xychart-beta
5959
title "MassiveParallelTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 3720
62-
bar [625.5, 1213, 3010.8, 3099.6, 132.3]
61+
y-axis "Time (ms)" 0 --> 3680
62+
bar [601.9, 1181.6, 2965.3, 3066.6, 131.6]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-12T21:13:20.517Z*
75+
*Last generated: 2025-11-13T00:28:35.381Z*

‎docs/docs/benchmarks/MatrixTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 5
77
# MatrixTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-12** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 557.69 ms | 556.60 ms | 4.221 ms |
20-
| NUnit | 4.4.0 | 1,539.35 ms | 1,541.87 ms | 11.514 ms |
21-
| MSTest | 4.0.2 | 1,496.25 ms | 1,496.55 ms | 5.576 ms |
22-
| xUnit3 | 3.2.0 | 1,586.79 ms | 1,586.67 ms | 7.339 ms |
23-
| **TUnit (AOT)** | 1.1.0 | 79.15 ms | 79.11 ms | 0.381 ms |
19+
| **TUnit** | 1.1.0 | 570.36 ms | 570.43 ms | 4.980 ms |
20+
| NUnit | 4.4.0 | 1,565.27 ms | 1,565.69 ms | 5.876 ms |
21+
| MSTest | 4.0.2 | 1,523.48 ms | 1,521.53 ms | 8.270 ms |
22+
| xUnit3 | 3.2.0 | 1,605.46 ms | 1,605.92 ms | 7.776 ms |
23+
| **TUnit (AOT)** | 1.1.0 | 79.85 ms | 79.80 ms | 0.324 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
5858
xychart-beta
5959
title "MatrixTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1905
62-
bar [557.69, 1539.35, 1496.25, 1586.79, 79.15]
61+
y-axis "Time (ms)" 0 --> 1927
62+
bar [570.36, 1565.27, 1523.48, 1605.46, 79.85]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-12T21:13:20.517Z*
75+
*Last generated: 2025-11-13T00:28:35.381Z*

‎docs/docs/benchmarks/ScaleTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 6
77
# ScaleTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-12** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 514.19 ms | 512.52 ms | 6.486 ms |
20-
| NUnit | 4.4.0 | 552.95 ms | 553.62 ms | 7.860 ms |
21-
| MSTest | 4.0.2 | 474.67 ms | 475.13 ms | 8.594 ms |
22-
| xUnit3 | 3.2.0 | 562.71 ms | 561.53 ms | 6.028 ms |
23-
| **TUnit (AOT)** | 1.1.0 | 44.36 ms | 45.13 ms | 3.857 ms |
19+
| **TUnit** | 1.1.0 | 516.15 ms | 515.58 ms | 7.373 ms |
20+
| NUnit | 4.4.0 | 582.62 ms | 580.89 ms | 8.879 ms |
21+
| MSTest | 4.0.2 | 564.54 ms | 562.45 ms | 7.780 ms |
22+
| xUnit3 | 3.2.0 | 578.98 ms | 576.98 ms | 9.057 ms |
23+
| **TUnit (AOT)** | 1.1.0 | 44.37 ms | 44.24 ms | 3.944 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
5858
xychart-beta
5959
title "ScaleTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 676
62-
bar [514.19, 552.95, 474.67, 562.71, 44.36]
61+
y-axis "Time (ms)" 0 --> 700
62+
bar [516.15, 582.62, 564.54, 578.98, 44.37]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-12T21:13:20.517Z*
75+
*Last generated: 2025-11-13T00:28:35.382Z*

‎docs/docs/benchmarks/SetupTeardownTests.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 7
77
# SetupTeardownTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-12** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-13** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,10 +16,10 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.1.0 | 583.4 ms | 583.7 ms | 5.16 ms |
20-
| NUnit | 4.4.0 | 1,198.4 ms | 1,195.0 ms | 15.35 ms |
21-
| MSTest | 4.0.2 | 1,162.5 ms | 1,161.5 ms | 9.11 ms |
22-
| xUnit3 | 3.2.0 | 1,243.3 ms | 1,241.9 ms | 9.80 ms |
19+
| **TUnit** | 1.1.0 | 562.2 ms | 561.5 ms | 4.24 ms |
20+
| NUnit | 4.4.0 | 1,173.5 ms | 1,173.5 ms | 9.02 ms |
21+
| MSTest | 4.0.2 | 1,146.7 ms | 1,145.3 ms | 16.63 ms |
22+
| xUnit3 | 3.2.0 | 1,216.8 ms | 1,217.2 ms | 5.86 ms |
2323
| **TUnit (AOT)** | 1.1.0 | NA | NA | NA |
2424

2525
## 📈 Visual Comparison
@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-12** from the latest CI
5858
xychart-beta
5959
title "SetupTeardownTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1492
62-
bar [583.4, 1198.4, 1162.5, 1243.3, 0]
61+
y-axis "Time (ms)" 0 --> 1461
62+
bar [562.2, 1173.5, 1146.7, 1216.8, 0]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-12T21:13:20.517Z*
75+
*Last generated: 2025-11-13T00:28:35.382Z*

‎docs/docs/benchmarks/index.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 1
77
# Performance Benchmarks
88

99
:::info Last Updated
10-
These benchmarks were automatically generated on **2025-11-12** from the latest CI run.
10+
These benchmarks were automatically generated on **2025-11-13** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -59,7 +59,7 @@ The benchmarks measure real-world testing patterns:
5959
- **Runtime**: .NET 10.0.0 (10.0.0, 10.0.25.52411), X64 RyuJIT x86-64-v3
6060
- **SDK**: .NET SDK 10.0.100
6161
- **Hardware**: GitHub Actions Standard Runner (Ubuntu)
62-
- **Tool**: BenchmarkDotNet v0.15.6, Linux Ubuntu 24.04.3 LTS (Noble Numbat)
62+
- **Tool**: BenchmarkDotNet v0.15.7, Linux Ubuntu 24.04.3 LTS (Noble Numbat)
6363

6464
### Why These Numbers Matter
6565

@@ -80,4 +80,4 @@ These benchmarks run automatically daily via [GitHub Actions](https://github.com
8080
Each benchmark runs multiple iterations with statistical analysis to ensure accuracy. Results may vary based on hardware and test characteristics.
8181
:::
8282

83-
*Last generated: 2025-11-12T21:13:20.518Z*
83+
*Last generated: 2025-11-13T00:28:35.383Z*

‎docs/static/benchmarks/AsyncTests.json‎

Lines changed: 22 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,51 +1,51 @@
11
{
2-
"timestamp": "2025-11-12T21:13:20.516Z",
2+
"timestamp": "2025-11-13T00:28:35.381Z",
33
"category": "AsyncTests",
44
"environment": {
5-
"benchmarkDotNetVersion": "BenchmarkDotNet v0.15.6, Linux Ubuntu 24.04.3 LTS (Noble Numbat)",
5+
"benchmarkDotNetVersion": "BenchmarkDotNet v0.15.7, Linux Ubuntu 24.04.3 LTS (Noble Numbat)",
66
"sdk": ".NET SDK 10.0.100",
77
"host": ".NET 10.0.0 (10.0.0, 10.0.25.52411), X64 RyuJIT x86-64-v3"
88
},
99
"results": [
1010
{
1111
"Method": "TUnit",
1212
"Version": "1.1.0",
13-
"Mean": "558.9 ms",
14-
"Error": "4.39 ms",
15-
"StdDev": "4.11 ms",
16-
"Median": "557.2 ms"
13+
"Mean": "551.3 ms",
14+
"Error": "2.23 ms",
15+
"StdDev": "1.86 ms",
16+
"Median": "551.1 ms"
1717
},
1818
{
1919
"Method": "NUnit",
2020
"Version": "4.4.0",
21-
"Mean": "650.2 ms",
22-
"Error": "10.15 ms",
23-
"StdDev": "9.00 ms",
24-
"Median": "649.0 ms"
21+
"Mean": "639.4 ms",
22+
"Error": "11.51 ms",
23+
"StdDev": "10.77 ms",
24+
"Median": "634.6 ms"
2525
},
2626
{
2727
"Method": "MSTest",
2828
"Version": "4.0.2",
29-
"Mean": "615.0 ms",
30-
"Error": "6.64 ms",
31-
"StdDev": "5.89 ms",
32-
"Median": "614.8 ms"
29+
"Mean": "612.9 ms",
30+
"Error": "10.30 ms",
31+
"StdDev": "9.13 ms",
32+
"Median": "611.2 ms"
3333
},
3434
{
3535
"Method": "xUnit3",
3636
"Version": "3.2.0",
37-
"Mean": "702.4 ms",
38-
"Error": "8.00 ms",
39-
"StdDev": "7.09 ms",
40-
"Median": "702.4 ms"
37+
"Mean": "692.8 ms",
38+
"Error": "6.56 ms",
39+
"StdDev": "5.81 ms",
40+
"Median": "692.6 ms"
4141
},
4242
{
4343
"Method": "TUnit_AOT",
4444
"Version": "1.1.0",
45-
"Mean": "123.8 ms",
46-
"Error": "0.49 ms",
47-
"StdDev": "0.46 ms",
48-
"Median": "123.9 ms"
45+
"Mean": "122.7 ms",
46+
"Error": "0.47 ms",
47+
"StdDev": "0.41 ms",
48+
"Median": "122.8 ms"
4949
}
5050
]
5151
}

0 commit comments

Comments
 (0)