Description
Using [NotInParallel] should prevent any other test from running during that test.
Per: https://tunit.dev/docs/execution/parallelism it states for NotInParallel:
If no keys are supplied, the test runs completely alone — no other test executes at the same time.
However this is violated if you use another test that depends on it with DependsOn both with and without ProceedOnFailure set to true.
Expected Behavior
The test always runs alone, if another test depends on it that test should run after it is done
Actual Behavior
The test runs with other tests that are not related running at the same time.
Steps to Reproduce
https://github.com/mitchcapper/__tunit_depends_bug repros it with two sample classes the only difference is one has a DependsOn and one does not.
It does seem to detect them correctly but still executes several tests during the one depended on:
Scheduling execution of 8 tests
Test 'DependsOnBug.TestsDepends.1.1.TestNoParallel.1.1.0': → NotInParallel
Test 'DependsOnBug.TestsDepends.1.1.TestFast.1.1.0': → Parallel (no constraints)
Test 'DependsOnBug.TestsDepends.1.1.TestNorm.1.1.0': → Parallel (no constraints)
Test 'DependsOnBug.TestsDepends.1.1.TestSlow.1.1.0': → Parallel (no constraints)
Test 'DependsOnBug.TestsNoDepends.1.1.TestNoParallel.1.1.0': → NotInParallel
Test 'DependsOnBug.TestsNoDepends.1.1.TestFast.1.1.0': → Parallel (no constraints)
Test 'DependsOnBug.TestsNoDepends.1.1.TestNorm.1.1.0': → Parallel (no constraints)
Test 'DependsOnBug.TestsNoDepends.1.1.TestSlow.1.1.0': → Parallel (no constraints)
═══ Test Grouping Summary ═══
Parallel (no constraints): 6 tests
ParallelGroups: 0 groups
ConstrainedParallelGroups: 0 groups
NotInParallel (global): 2 tests
KeyedNotInParallel: 0 tests
════════════════════════════
Starting 6 parallel tests
failed TestNoParallel (17ms)
[Assertion Failure] Expected to be 1
but found 3
at Assert.That(cntr).IsEqualTo(1)
total: 8
failed: 1
succeeded: 7
skipped: 0
duration: 11s 239ms
TUnit Version
1.41
.NET Version
.net 10
Operating System
Windows
IDE / Test Runner
dotnet CLI (dotnet test / dotnet run)
Error Output / Stack Trace
Additional Context
This came up as I was testing an MCP connection tool. One thing the tool does is it is meant to gracefully handle server restarts without the client knowing the server restarted. One test terminates the server connection to test this, then it makes sure a request after the break properly restarts a session. The code works but if any requests are happening to the server during this time, clearly they fail as the connection is terminated. In actual use this is no problem, there are no requests happening with then the server restarts, but for testing clearly it runs many tests at once (and I specifically have them share one connection to test the parallelism).
The NotInParallel attribute on the test that breaks the connection fixes the tests possibly failing if it breaks during another test. The problem came when I then wanted a different test that tested notifications to run after the NotInParallel test to make sure notifications were also resumed correctly. As soon as I added the DependsOn testing violated the NotInParallel attribute causing tests to fail.
Without having to add attributes to all the NotInParallel and without naming every other test in an [After] attribute I didn't see a work around to make it execute the one test after all the other tests (except for the one test that depended on it).
IDE-Specific Issue?
Description
Using
[NotInParallel]should prevent any other test from running during that test.Per: https://tunit.dev/docs/execution/parallelism it states for NotInParallel:
If no keys are supplied, the test runs completely alone — no other test executes at the same time.However this is violated if you use another test that depends on it with
DependsOnboth with and without ProceedOnFailure set to true.Expected Behavior
The test always runs alone, if another test depends on it that test should run after it is done
Actual Behavior
The test runs with other tests that are not related running at the same time.
Steps to Reproduce
https://github.com/mitchcapper/__tunit_depends_bug repros it with two sample classes the only difference is one has a DependsOn and one does not.
It does seem to detect them correctly but still executes several tests during the one depended on:
TUnit Version
1.41
.NET Version
.net 10
Operating System
Windows
IDE / Test Runner
dotnet CLI (dotnet test / dotnet run)
Error Output / Stack Trace
Additional Context
This came up as I was testing an MCP connection tool. One thing the tool does is it is meant to gracefully handle server restarts without the client knowing the server restarted. One test terminates the server connection to test this, then it makes sure a request after the break properly restarts a session. The code works but if any requests are happening to the server during this time, clearly they fail as the connection is terminated. In actual use this is no problem, there are no requests happening with then the server restarts, but for testing clearly it runs many tests at once (and I specifically have them share one connection to test the parallelism).
The
NotInParallelattribute on the test that breaks the connection fixes the tests possibly failing if it breaks during another test. The problem came when I then wanted a different test that tested notifications to run after theNotInParalleltest to make sure notifications were also resumed correctly. As soon as I added the DependsOn testing violated theNotInParallelattribute causing tests to fail.Without having to add attributes to all the
NotInParalleland without naming every other test in an [After] attribute I didn't see a work around to make it execute the one test after all the other tests (except for the one test that depended on it).IDE-Specific Issue?
dotnet testordotnet run, not just in my IDE