Skip to content

[wasm] Add support for a random test case orderer, for xunit tests #65628

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
radical merged 8 commits into from
Feb 21, 2022
Merged

[wasm] Add support for a random test case orderer, for xunit tests #65628

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ce4eef9
[wasm] Add support for a random test case orderer, for xunit tests
radical Feb 20, 2022
9b5fb03
Enable test orderer only for libraries tests
radical Feb 20, 2022
3bb53db
[wasm] Automatically pass XUNIT_RANDOM_ORDER_SEED envvar to wasm apps
radical Feb 20, 2022
d17c804
Disable random test ordering for `System.Xml.RW.XmlWriterApi.Tests`
radical Feb 20, 2022
0752bed
[wasm] Disable random tests ordering for `System.Runtime.Loader`
radical Feb 20, 2022
3fc14be
[wasm] Disable random test ordering for `System.Runtime.Numerics`
radical Feb 20, 2022
c924446
Add issues for the failing tests
radical Feb 21, 2022
153a0d1
Merge remote-tracking branch 'origin/main' into tests-random-order
radical Feb 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[wasm] Disable random test ordering for System.Runtime.Numerics 
Randomized runs seem to fail with:

```
[03:29:54] info: Starting:    System.Runtime.Numerics.Tests.dll
[03:29:58] fail: [FAIL] System.Numerics.Tests.cast_toTest.RunDoubleExplicitCastToBigIntegerTests
[03:29:58] info: Assert.Equal() Failure
[03:29:58] info:           ↓ (pos 0)
[03:29:58] info: Expected: -0
[03:29:58] info: Actual:   0
[03:29:58] info:           ↑ (pos 0)
[03:29:58] info:    at System.Numerics.Tests.cast_toTest.VerifyDoubleExplicitCastToBigInteger(Double value)
[03:29:58] info:    at System.Numerics.Tests.cast_toTest.RunDoubleExplicitCastToBigIntegerTests()
[03:29:58] info:    at System.Reflection.RuntimeMethodInfo.InvokeWorker(Object obj, BindingFlags invokeAttr, Span`1 parameters)
```

Reproducible with `XUNIT_RANDOM_SEED_ORDER=1883302047`.
  • Loading branch information
@radical
radical authored Feb 20, 2022
commit 3fc14be877dff93b217044f6135b1d271ed75387
1 change: 1 addition & 0 deletions src/libraries/System.Runtime.Numerics/tests/System.Runtime.Numerics.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
<PropertyGroup>
<IncludeRemoteExecutor>true</IncludeRemoteExecutor>
<TargetFrameworks>$(NetCoreAppCurrent)</TargetFrameworks>
<XUnitUseRandomizedTestOrderer>false</XUnitUseRandomizedTestOrderer>
</PropertyGroup>
<ItemGroup>
<Compile Include="BigInteger\absolutevalue.cs" />
Expand Down