This repository was archived by the owner on Jan 23, 2023. It is now read-only.
Add more Timer tests #32729
Merged
Add more Timer tests #32729
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
Author
|
@dotnet-bot test Outerloop Windows x64 Debug Build please |
Member
Author
|
@dotnet-bot test Outerloop Windows x64 Debug Build please |
Member
Author
|
This is interesting, and @kouvel, it may warrant some follow-up. Note that this is all happening before my changes in dotnet/coreclr#20302, which haven't been consumed into corefx yet. For now, I'm going to disable the affected test on macOS in order to get this PR merged. This PR adds several new tests for Timer, but most importantly, it adds a test that runs a bunch of single-fire timers and measures how long they take to fire. For each timer, it allows for a 50ms buffer. If at least 90% of the timers all complete within that buffer from their expected due time, the test passes; if more than 10% don't, the test fails. On Windows and Linux, they've all passed every time CIs run them. On macOS, all but one of the InlineDatas has failed every time CIs run them.
I augmented the test to output diagnostic data in case of failure, so that for each dueTime, we can see what the actual failing times were. They're below: TimerFiringTests/Timer_ManyDifferentSerialSingleDueTimes_AllFireWithinAllowedRange(dueTimes: [1000]): Timer_ManyDifferentSerialSingleDueTimes_AllFireWithinAllowedRange(dueTimes: [15]): and Timer_ManyDifferentSerialSingleDueTimes_AllFireWithinAllowedRange(dueTimes: [200, 250, 300]): and Timer_ManyDifferentSerialSingleDueTimes_AllFireWithinAllowedRange(dueTimes: [200, 300, 400]): and Timer_ManyDifferentSerialSingleDueTimes_AllFireWithinAllowedRange(dueTimes: [332, 333, 334]): and Timer_ManyDifferentSerialSingleDueTimes_AllFireWithinAllowedRange(dueTimes: [333]): and Timer_ManyDifferentSerialSingleDueTimes_AllFireWithinAllowedRange(dueTimes: [400, 450, 500]): and There could of course be reasonable explanations for this. Maybe the macOS devices are more underpowered than the Windows and Linux VMs used in CI. Maybe they're more affected by other tests running concurrently in CI. Maybe the machines were busy doing other things as well, more so than for Windows and Linux, and is skewing things. My concern is that maybe there actually is something in either the PAL or the thread pool on macOS that's contributing. |
Exercise scenarios involving lots of timers.
The test is failing with very high percentages (e.g. 100%) on macOS in CI. Add more data to find out why.
stephentoub
force-pushed
the
timertests
branch
from
f712f3c to
3bc14f8
Compare
Member
|
And it doesn't happen on your local Mac? |
Member
Author
|
I haven't tried yet locally. I managed to get my mac into a bad state, and I need to reimage it. If you have a few cycles to try, that'd be interesting. (Note that in the last commit, in addition to disabling the test, I also made it a bit more forgiving, increasing both the buffer and the allowed-failed percentage. So if you do try locally, just delete the last commit.) |
Member
Author
|
@dotnet-bot test Tizen armel Debug Build please |
picenka21
pushed a commit
to picenka21/runtime
that referenced
this pull request
Feb 18, 2022
* Add more Timer tests Exercise scenarios involving lots of timers. * Add debugging details to new test The test is failing with very high percentages (e.g. 100%) on macOS in CI. Add more data to find out why. * Disable test on macOS, and make test a little more forgiving, too Commit migrated from dotnet/corefx@61d06d8
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Exercise scenarios involving lots of timers.
cc: @vancem, @kouvel