Skip to content

[Perf] Changes at 7/22/2022 1:57:08 AM #6861

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 57469b2781baecc6e8844a60993005a7185a34b4
Compare 0c32267664e1b9913f2329827d998646fecafd91
Diff Diff

Improvements in System.ComponentModel.Tests.Perf_TypeDescriptorTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetConverter - Duration of single invocation 284.84 ns 253.46 ns 0.89 0.05 False
GetConverter - Duration of single invocation 219.07 ns 192.01 ns 0.88 0.09 False
GetConverter - Duration of single invocation 216.99 ns 193.37 ns 0.89 0.07 False
GetConverter - Duration of single invocation 124.18 ns 108.02 ns 0.87 0.09 False
GetConverter - Duration of single invocation 146.37 ns 124.46 ns 0.85 0.09 False
GetConverter - Duration of single invocation 214.70 ns 191.57 ns 0.89 0.04 False
GetConverter - Duration of single invocation 220.20 ns 198.89 ns 0.90 0.07 False
GetConverter - Duration of single invocation 222.17 ns 194.60 ns 0.88 0.09 False
GetConverter - Duration of single invocation 148.00 ns 123.44 ns 0.83 0.11 False
GetConverter - Duration of single invocation 148.35 ns 123.83 ns 0.83 0.09 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'
Details

Payloads

Baseline
Compare

Histogram

System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeEnum))


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 253.4570450920396 < 261.3300194686824.
IsChangePoint: Marked as a change because one of 5/28/2022 3:04:49 PM, 7/7/2022 7:47:15 PM, 7/9/2022 5:50:00 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 48.35525816744614 (T) = (0 -249.93684999316378) / Math.Sqrt((69.73760139852493 / (299)) + (4.204084486581822 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.10919090662594981 = (280.57285433234284 - 249.93684999316378) / 280.57285433234284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeValueType?))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 192.01372596080378 < 204.96111502102326.
IsChangePoint: Marked as a change because one of 7/7/2022 7:47:15 PM, 7/9/2022 5:50:00 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 40.55761029586028 (T) = (0 -193.76327224804425) / Math.Sqrt((63.00917386870028 / (248)) + (3.9931805648012837 / (25))) is greater than 1.9687563138230615 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (248) + (25) - 2, .975) and 0.11866804953607143 = (219.85277187108474 - 193.76327224804425) / 219.85277187108474 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Guid))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 193.3693013785647 < 205.36491611333088.
IsChangePoint: Marked as a change because one of 5/28/2022 6:51:39 AM, 7/7/2022 7:47:15 PM, 7/9/2022 2:13:52 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 51.139001512862926 (T) = (0 -193.70906173256515) / Math.Sqrt((54.54702328216281 / (299)) + (2.4570547345842484 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.12270934013520547 = (220.80374338240307 - 193.70906173256515) / 220.80374338240307 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.IDerived))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.02114867718669 < 117.42943586262076.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 64.07175339458034 (T) = (0 -108.43502929337333) / Math.Sqrt((14.427492542741064 / (299)) + (0.5111713505125199 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.13410327372750122 = (125.22859366862728 - 108.43502929337333) / 125.22859366862728 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassIDerived))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 124.45893988375181 < 139.58846913350357.
IsChangePoint: Marked as a change because one of 6/1/2022 3:16:04 PM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 64.42641451020074 (T) = (0 -125.28354846674621) / Math.Sqrt((33.21170853735641 / (299)) + (1.0785698364940806 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.168828456787627 = (150.73127742384096 - 125.28354846674621) / 150.73127742384096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Enum))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 191.57137904420074 < 203.90131556063574.
IsChangePoint: Marked as a change because one of 5/28/2022 3:04:49 PM, 7/7/2022 7:47:15 PM, 7/9/2022 5:50:00 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 54.26490738881808 (T) = (0 -193.22162319486765) / Math.Sqrt((60.5661778517751 / (299)) + (1.293995569430802 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.12405953341067442 = (220.5876204660577 - 193.22162319486765) / 220.5876204660577 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(int?))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 198.88895281799498 < 211.34033566256144.
IsChangePoint: Marked as a change because one of 7/7/2022 7:47:15 PM, 7/9/2022 5:50:00 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 52.31774086363684 (T) = (0 -200.1210137280621) / Math.Sqrt((52.4818895438079 / (299)) + (2.493879627059702 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.12062019504183201 = (227.57062716215304 - 200.1210137280621) / 227.57062716215304 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.DerivedClass))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 194.59611971655002 < 205.19232794473461.
IsChangePoint: Marked as a change because one of 4/20/2022 12:55:12 PM, 5/25/2022 3:58:08 AM, 7/7/2022 7:47:15 PM, 7/9/2022 5:50:00 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 47.05748906337407 (T) = (0 -194.50845894437992) / Math.Sqrt((93.55764371099177 / (299)) + (1.2390580846616277 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1271362095212529 = (222.8394178634632 - 194.50845894437992) / 222.8394178634632 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(string))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 123.43672426282079 < 140.00447490214046.
IsChangePoint: Marked as a change because one of 5/25/2022 3:58:08 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 60.67702647965596 (T) = (0 -124.56943901516271) / Math.Sqrt((30.667183989479916 / (299)) + (1.8364642964077849 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.16968456333853324 = (150.0266447123176 - 124.56943901516271) / 150.0266447123176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassWithNoConverter))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 123.83318271162689 < 140.6201000300916.
IsChangePoint: Marked as a change because one of 5/25/2022 8:13:38 AM, 7/21/2022 8:47:55 PM, 7/28/2022 6:13:35 AM falls between 7/19/2022 2:39:18 PM and 7/28/2022 6:13:35 AM.
IsImprovementStdDev: Marked as improvement because 68.59982747030897 (T) = (0 -125.23771524102622) / Math.Sqrt((24.46623418615821 / (299)) + (1.4606609352455695 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.17021896583394136 = (150.92863066807962 - 125.23771524102622) / 150.92863066807962 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions