[Perf] Changes at 5/31/2022 9:21:33 PM #5796
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 6010dc0604a9f2708dba8d21f57d97b613dfcabe |
| Compare | 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377 |
| Diff | Diff |
Improvements in System.MathBenchmarks.Double
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Min - Duration of single invocation | 7.28 μs | 6.43 μs | 0.88 | 0.14 | False | |||||
| CopySign - Duration of single invocation | 12.07 μs | 10.91 μs | 0.90 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'Details
Payloads
Histogram
System.MathBenchmarks.Double.Min
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.431013818831441 < 6.625204513262238.
IsChangePoint: Marked as a change because one of 5/1/2022 6:14:18 PM, 5/11/2022 3:41:15 PM, 5/30/2022 12:57:05 PM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 15.324753738383722 (T) = (0 -6361.583955178732) / Math.Sqrt((64547.13524337057 / (299)) + (27774.125496029723 / (40))) is greater than 1.9670282846691574 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (40) - 2, .975) and 0.06775390771526732 = (6823.932015191259 - 6361.583955178732) / 6823.932015191259 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.MathBenchmarks.Double.CopySign
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.914098351456504 < 11.485232975705289.
IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 74.34345967646577 (T) = (0 -10914.219330189238) / Math.Sqrt((51538.79782015337 / (299)) + (0.14221354987212387 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.0820892453572769 = (11890.28375033841 - 10914.219330189238) / 11890.28375033841 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 6010dc0604a9f2708dba8d21f57d97b613dfcabe |
| Compare | 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377 |
| Diff | Diff |
Improvements in System.Collections.CtorFromCollectionNonGeneric<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Queue - Duration of single invocation | 9.19 μs | 6.67 μs | 0.73 | 0.06 | True | |||||
| Stack - Duration of single invocation | 7.86 μs | 6.33 μs | 0.80 | 0.04 | True |
_1.png)
_2.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollectionNonGeneric<String>*'Details
Payloads
Histogram
System.Collections.CtorFromCollectionNonGeneric<String>.Queue(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.673495306776557 < 8.34418277676437.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 92.83279322592477 (T) = (0 -6544.207990250077) / Math.Sqrt((80662.09507141617 / (299)) + (8671.903805604383 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.23639118998410644 = (8570.1054052976 - 6544.207990250077) / 8570.1054052976 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Collections.CtorFromCollectionNonGeneric<String>.Stack(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.325053094660195 < 7.506594930512328.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 75.2199759570073 (T) = (0 -6503.900289340192) / Math.Sqrt((48916.32806851793 / (299)) + (6280.336715912395 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.16910093357918335 = (7827.545549372697 - 6503.900289340192) / 7827.545549372697 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 6010dc0604a9f2708dba8d21f57d97b613dfcabe |
| Compare | 8a043bf7adb0fbf5e60a8dd557c98686bc0a8377 |
| Diff | Diff |
Improvements in System.MathBenchmarks.Single
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CopySign - Duration of single invocation | 11.90 μs | 10.91 μs | 0.92 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'Details
Payloads
Histogram
System.MathBenchmarks.Single.CopySign
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.913880868607649 < 11.31879097661773.
IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 82.96716387881976 (T) = (0 -10914.303884951278) / Math.Sqrt((39663.22658750866 / (299)) + (0.8723586247888608 / (41))) is greater than 1.9670073106672175 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (41) - 2, .975) and 0.08051010907050839 = (11869.955279136624 - 10914.303884951278) / 11869.955279136624 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository