[4.6.166] 0 visitedSequencePoint while this is obviously false

[Greybird]

Hi,

On my software factory, I get quite consistenly 0 visitedSequencePoint, hence 0% coverage, which is wrong, as even the passed tests are considered unvisited.

I did try -register:user, pregesister using regsvr32 (and I can confirm keys in the registry are present in HKLM).

I run the opencovers in parallel on separate assemblies under jenkins, using the following command line:

D:/Jenkins/Tools/OpenCover-4.6.166/OpenCover.Console.exe 
  -hideskipped:All 
  -mergebyhash 
  -skipautoprops 
  -threshold:1 
  -target:D:/Jenkins/Tools/NUnit/bin/nunit-console-x86.exe 
  -targetargs:-framework:net-4.0 -nologo -noshadow -labels -work=D:\Jenkins\workspace\SONAR-Coverage\NUnit-report -result=TestAssembly-1.xml -include:-GUI-nonreg TestAssembly.dll 
  -output:D:\Jenkins\workspace\SONAR-Coverage\OpenCover-report\TestAssembly-1.xml

I tried to make the jenkins account admin of the machine, but it did not change.

On the 200+ jobs running in parallel (at around 20 max at the same time), around 70% of them are getting 0% coverage.
When trying to run only a single assembly at once, I also get 0% coverage.

When I run the coverage on a single assembly alone on my dev pc, I get a coverage without issue.

Could you please help in figuring out what happens ? Do I miss a config ? Should you need a debug version of opencover to get more info, I can totally install it and provide you logs.

Thanks in advance

Arnaud

[sawilde]

What information is in the XML file regarding the assemblies?

[Greybird]

Hi,

File starts like this:

<?xml version="1.0" encoding="utf-8"?>
<CoverageSession xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Summary numSequencePoints="419525" visitedSequencePoints="0" numBranchPoints="202070" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="419" minCyclomaticComplexity="1" visitedClasses="0" numClasses="7429" visitedMethods="0" numMethods="70522" />
  <Modules>
    <Module hash="07-0C-D8-5F-DB-CC-EB-88-B4-E2-2D-F1-6B-B8-5C-A6-6E-8A-9E-0D">
      <Summary numSequencePoints="956" visitedSequencePoints="0" numBranchPoints="183" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="14" minCyclomaticComplexity="1" visitedClasses="0" numClasses="10" visitedMethods="0" numMethods="99" />
      <FullName>D:\Jenkins\workspace\SONAR-Coverage\src\bin\Tests\Debug\TestAssembly.dll</FullName>
      <ModuleName>TestAssembly</ModuleName>
      <Files>

[sawilde]

I think you haven't passed the arguments to OpenCover/Nunit correctly. You need to escape wrap all the arguments to NUnit (aka passed via targetargs) with quotes e.g.

"-targetargs:-framework:net-4.0 -nologo -noshadow -labels -work=D:\Jenkins\workspace\SONAR-Coverage\NUnit-report -result=TestAssembly-1.xml -include:-GUI-nonreg TestAssembly.dll"

I have made an update to OpenCover itself to be more strict on arguments it does not recognise

[Greybird]

Thanks a lot for your interest in my issue!

I checked on another chain (as the one I encounter the problem is not available for the night), and we seem to run with the adequate quotes.

However, you change cannot do any harm from my point of view.

I will check tomorrow on the problematic chain to make sure, and let you know my findings.

I cannot share code as explained, but I think I may probably get permission to export test results and coverage report if needed.
Also I could be able to export some kind of procmon activity log if this could be of help ?.

Feel free to tell me if this would be of interest, as well as any other info you would need.
In this case, I would check with our legal department what is doable.

Arnaud

[Greybird]

Hi,

Sorry for the delay.

I was able to check, and found that arguments are correctly passed.

I was wondering if the fact that I use opencover to monitor x86 nunit console process which can in turn launch some x86 or x64 processes could be a problem ?

When I launch a unitary assembly of a x86 console running a x64 executable, both of them are covered correctly.But could there be an issue with this way of doing ?
Meanwhile, I'll try to make both nunit and all ran processes x64 to see if something changes.

Arnaud

[sawilde]

OpenCover should be able to handle both 32 and 64 bit processes at the same, each profiler instance, one per .NET process, should create its own shared memory buffers with the host process. The issue may be the number of buffers being allocated at any one time but you say you have seen the issue even when just one test run is in action so I doubt think this is why you getting 0% coverage.

Also I could be able to export some kind of procmon activity log if this could be of help ?.

I am not sure what is in such a log to know whether it would be of any use but I doubt it would capture any internal behaviour e.g. instrumentation.

For a single test run that produces 0% it would be interesting to see the coverage report XML - what would be nice is to give me a copy of a repeatable sample (DLLs/PDBs) of code and tests, this may be possible with a small subset of code that has low value but still repeats the problem. I don't know what your product is to know if this is possible.

[Greybird]

Hi,

I'll try to find a repeatable use case then, and will let you know if i achieve this.

Thanks !

Arnaud

[sawilde]

I think I have found a race condition that may account for your issue - I'll be pushing out a release candidate hopefully in a few days - I'll let you know more

[Greybird]

Thanks a lot !

This is great news as I didn't manage to create a repro yet.
I'll test you RC once you will provide it.

[sawilde]

A build is available on AppVeyor - it's part of a issue resolution for another defect - https://ci.appveyor.com/project/sawilde/opencover/build/artifacts