Skip to content

Conversation

@stephentoub
Copy link
Member

@stephentoub stephentoub commented May 5, 2020

Fixes #35807

With dotnet test turned on, xunit is now pre-enumerating all theory data. This is causing two problems:

  1. Most of our theories that use MemberData are causing useless warnings to be output to the console when running tests. For a test suite like PLINQ's, this results in ~170KB of warning text spewed.
  2. The pre-enumeration is adding non-trivial time overhead to test execution, e.g. on my machine System.Text.RegularExpressions.Tests are now taking > 15 seconds to execute whereas previously they took ~3 seconds.

The downside of this is the VS test explorer will group all data rows for a theory as a single item, but it's already doing that for many of our theories because they're not IXunitSerializable. And someone who wants to debug just a single item can comment out the rest.

@ghost
Copy link

ghost commented May 5, 2020

Tagging subscribers to this area: @ViktorHofer
Notify danmosemsft if you want to be subscribed.

@ghost
Copy link

ghost commented May 5, 2020

Hello @ViktorHofer!

Because this pull request has the auto-merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

@stephentoub
Copy link
Member Author

stephentoub commented May 5, 2020

And just in case anyone complains, here's an example of why this needs to be turned off:

Running System.IO.FileSystem.Tests...

Before:

 Total time: 3.8580 Minutes

After:

 Total time: 15.5734 Seconds

@ghost ghost merged commit 0383079 into dotnet:master May 5, 2020
@stephentoub stephentoub deleted the disablepreenumerate branch October 12, 2020 00:28
@ghost ghost locked as resolved and limited conversation to collaborators Dec 9, 2020
This pull request was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

X509Certificates.Tests: Non-serializable data ('System.Object[]')

5 participants