IncludeTestAssembly should default to true

[sharwell]

I just enabled coverlet on a new repository, and the coverage reports were incomplete. The root cause turned out to be a setting IncludeTestAssembly, which confusingly excluded code from the coverage report even though that code was not part of any of the various exclude properties.

Test code is an essential part of a production code base, and should always be included in coverage collection. It's fine to provide a property so it can be disabled, but the property should default to the value users are expected to use.

[MarcoRossignoli]

We did this discussion a lot of times

#372 (comment)
#318 (comment)

and also on other issues...

Personally I don't have a strong opinions...I observed that a lot of users expect that test project won't be included in coverage by design, they don't see test project useful to cover but that add noise to results.

Test code is an essential part of a production code base

I agree, this perfect "accademic" statement makes a lot of sense...btw there is a drift between theory and reality for what I observed.

Make a change now could be a "breaking" in expected result, I mean coverage thresholds will raise and maybe they will lose possible coverage issue.

Said that, we can change flag and defaults and announce a breaking in behaviour, it's doable.

I don't want to be only voice here, I can implement the change but I would like to reach a more large agreement once and for all if possible, so I ask to more "active" users what they think about, also because after vstest integration coverlet is the "xplat coverage tool" and stuff are becoming interesting https://nugettrends.com/packages?months=12&ids=coverlet.msbuild&ids=coverlet.collector&ids=coverlet.console

@tonerdo @petli @ViktorHofer @vagisha-nidhi(vstest) @AArnott and if you want to cc someone else that could help please do.

OOT I sell always vstest integration like the best way to consume coverlet, due to well know issue https://github.com/tonerdo/coverlet/blob/master/Documentation/KnowIssues.md

[sharwell]

It's been a while so I forgot I was part of the discussion. 🤦‍♂

I just filed the issue since I was switching a repository from OpenCover to Coverlet and observed the unexpected lack of coverage in test code. I was setting up coverage in large part to help eliminate dead test code and find cases where tests were not running in the expected manner.

[AArnott]

I would like coverage data on test projects by default, but IMO the % coverage reports should exclude the test projects. If we strive for 90% coverage on the product assembly, getting 98% on the test assembly shouldn't count toward the 90% goal on the product.

[PureKrome]

I know i'm late to this party - so I'm trying to understand what people are thinking here.

A test method tests some product assembly/method. Kewl. So why do we want coverage on a test method? The tests are there to verify the product assembly is working great. The coverage is used to make sure the product assembly is tested appropriately.

I don't see why anyone would care if a test method is ... covered? All I could see that this could highlight are some 'orphaned' methods in the test project which are called/used any more?

[MarcoRossignoli]

Sam's comment #318 (comment)

[MarcoRossignoli]

Seems that exclude test project by default is the expected behavior for the majority.