-
Notifications
You must be signed in to change notification settings - Fork 3
Closed
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 57469b2781baecc6e8844a60993005a7185a34b4 |
| Compare | 0c32267664e1b9913f2329827d998646fecafd91 |
| Diff | Diff |
Improvements in System.ComponentModel.Tests.Perf_TypeDescriptorTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| GetConverter - Duration of single invocation | 284.84 ns | 253.46 ns | 0.89 | 0.05 | False | |||||
| GetConverter - Duration of single invocation | 219.07 ns | 192.01 ns | 0.88 | 0.09 | False | |||||
| GetConverter - Duration of single invocation | 216.99 ns | 193.37 ns | 0.89 | 0.07 | False | |||||
| GetConverter - Duration of single invocation | 124.18 ns | 108.02 ns | 0.87 | 0.09 | False | |||||
| GetConverter - Duration of single invocation | 146.37 ns | 124.46 ns | 0.85 | 0.09 | False | |||||
| GetConverter - Duration of single invocation | 214.70 ns | 191.57 ns | 0.89 | 0.04 | False | |||||
| GetConverter - Duration of single invocation | 220.20 ns | 198.89 ns | 0.90 | 0.07 | False | |||||
| GetConverter - Duration of single invocation | 222.17 ns | 194.60 ns | 0.88 | 0.09 | False | |||||
| GetConverter - Duration of single invocation | 148.00 ns | 123.44 ns | 0.83 | 0.11 | False | |||||
| GetConverter - Duration of single invocation | 148.35 ns | 123.83 ns | 0.83 | 0.09 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'Details
Payloads
Histogram
System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeEnum))
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 253.4570450920396 < 261.3300194686824.
IsChangePoint: Marked as a change because one of 5/28/2022 3:04:49 PM, 7/7/2022 7:47:15 PM, 7/9/2022 5:50:00 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 48.35525816744614 (T) = (0 -249.93684999316378) / Math.Sqrt((69.73760139852493 / (299)) + (4.204084486581822 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.10919090662594981 = (280.57285433234284 - 249.93684999316378) / 280.57285433234284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeValueType?))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 192.01372596080378 < 204.96111502102326.
IsChangePoint: Marked as a change because one of 7/7/2022 7:47:15 PM, 7/9/2022 5:50:00 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 40.55761029586028 (T) = (0 -193.76327224804425) / Math.Sqrt((63.00917386870028 / (248)) + (3.9931805648012837 / (25))) is greater than 1.9687563138230615 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (248) + (25) - 2, .975) and 0.11866804953607143 = (219.85277187108474 - 193.76327224804425) / 219.85277187108474 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Guid))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 193.3693013785647 < 205.36491611333088.
IsChangePoint: Marked as a change because one of 5/28/2022 6:51:39 AM, 7/7/2022 7:47:15 PM, 7/9/2022 2:13:52 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 51.139001512862926 (T) = (0 -193.70906173256515) / Math.Sqrt((54.54702328216281 / (299)) + (2.4570547345842484 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.12270934013520547 = (220.80374338240307 - 193.70906173256515) / 220.80374338240307 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.IDerived))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.02114867718669 < 117.42943586262076.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 64.07175339458034 (T) = (0 -108.43502929337333) / Math.Sqrt((14.427492542741064 / (299)) + (0.5111713505125199 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.13410327372750122 = (125.22859366862728 - 108.43502929337333) / 125.22859366862728 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassIDerived))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 124.45893988375181 < 139.58846913350357.
IsChangePoint: Marked as a change because one of 6/1/2022 3:16:04 PM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 64.42641451020074 (T) = (0 -125.28354846674621) / Math.Sqrt((33.21170853735641 / (299)) + (1.0785698364940806 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.168828456787627 = (150.73127742384096 - 125.28354846674621) / 150.73127742384096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Enum))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 191.57137904420074 < 203.90131556063574.
IsChangePoint: Marked as a change because one of 5/28/2022 3:04:49 PM, 7/7/2022 7:47:15 PM, 7/9/2022 5:50:00 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 54.26490738881808 (T) = (0 -193.22162319486765) / Math.Sqrt((60.5661778517751 / (299)) + (1.293995569430802 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.12405953341067442 = (220.5876204660577 - 193.22162319486765) / 220.5876204660577 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(int?))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 198.88895281799498 < 211.34033566256144.
IsChangePoint: Marked as a change because one of 7/7/2022 7:47:15 PM, 7/9/2022 5:50:00 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 52.31774086363684 (T) = (0 -200.1210137280621) / Math.Sqrt((52.4818895438079 / (299)) + (2.493879627059702 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.12062019504183201 = (227.57062716215304 - 200.1210137280621) / 227.57062716215304 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.DerivedClass))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 194.59611971655002 < 205.19232794473461.
IsChangePoint: Marked as a change because one of 4/20/2022 12:55:12 PM, 5/25/2022 3:58:08 AM, 7/7/2022 7:47:15 PM, 7/9/2022 5:50:00 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 47.05748906337407 (T) = (0 -194.50845894437992) / Math.Sqrt((93.55764371099177 / (299)) + (1.2390580846616277 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1271362095212529 = (222.8394178634632 - 194.50845894437992) / 222.8394178634632 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(string))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 123.43672426282079 < 140.00447490214046.
IsChangePoint: Marked as a change because one of 5/25/2022 3:58:08 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 60.67702647965596 (T) = (0 -124.56943901516271) / Math.Sqrt((30.667183989479916 / (299)) + (1.8364642964077849 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.16968456333853324 = (150.0266447123176 - 124.56943901516271) / 150.0266447123176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassWithNoConverter))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 123.83318271162689 < 140.6201000300916.
IsChangePoint: Marked as a change because one of 5/25/2022 8:13:38 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 68.59982747030897 (T) = (0 -125.23771524102622) / Math.Sqrt((24.46623418615821 / (299)) + (1.4606609352455695 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.17021896583394136 = (150.92863066807962 - 125.23771524102622) / 150.92863066807962 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository









