-
Notifications
You must be signed in to change notification settings - Fork 398
Expand file tree
/
Copy pathREADME.md
More file actions
260 lines (162 loc) · 11.6 KB
/
README.md
File metadata and controls
260 lines (162 loc) · 11.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
# Coverlet
[](https://dev.azure.com/tonerdo/coverlet/_build/latest?definitionId=5&branchName=master)  [](https://github.com/coverlet-coverage/coverlet/blob/master/LICENSE)
| Driver | Current version | Downloads |
| --- | --- | --- |
| coverlet.MTP | [](https://www.nuget.org/packages/coverlet.MTP/) | [](https://www.nuget.org/packages/coverlet.MTP/) |
| coverlet.collector | [](https://www.nuget.org/packages/coverlet.collector/) | [](https://www.nuget.org/packages/coverlet.collector/)
| coverlet.msbuild | [](https://www.nuget.org/packages/coverlet.msbuild/) | [](https://www.nuget.org/packages/coverlet.msbuild/) |
| coverlet.console | [](https://www.nuget.org/packages/coverlet.console/) | [](https://www.nuget.org/packages/coverlet.console/) |
Coverlet is a cross platform code coverage framework for .NET, with support for line, branch and method coverage. It works with [.NET Framework](Documentation/KnownIssues.md#badimageformatexception-net-framework-47x-48x) on Windows and .NET Core on all supported platforms.
> [!NOTE]
> Coverlet only supports modern .NET [SDK-style](https://learn.microsoft.com/en-us/dotnet/core/project-sdk/overview) projects.
**Coverlet documentation reflect the current repository state of the features, not the released ones.**
**Check the [changelog](Documentation/Changelog.md) to understand if the documented feature you want to use has been officially released.**
## Main contents
* [QuickStart](#quick-start)
* [How It Works](#how-it-works)
* [Drivers features differences](Documentation/DriversFeatures.md)
* [Deterministic build support](#deterministic-build-support)
* [Known Issues](#known-issues)
* [Consume nightly build](#consume-nightly-build)
* [Feature samples](Documentation/Examples.md)
* [Cake Add-In](#cake-add-in)
* [Visual Studio Add-In](#visual-studio-add-in)
* [Changelog](Documentation/Changelog.md)
* [Roadmap](Documentation/Roadmap.md)
## Quick Start
Coverlet can be used through four different *drivers*
* VSTest engine integration
* MSBuild task integration
* As a .NET Global tool (supports standalone integration tests)
* **New** Microsoft Testing Platform integration (coverlet.MTP)
Coverlet supports only SDK-style projects <https://docs.microsoft.com/en-us/visualstudio/msbuild/how-to-use-project-sdk?view=vs-2019>
### **NEW** Microsoft Testing Platform Integration (coverlet.MTP)
### Installation (coverlet.MTP)
```bash
dotnet add package coverlet.MTP
```
> [!NOTE]
> Add the `coverlet.MTP` package only to test projects that use the [Microsoft Testing Platform](https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-intro). This package is designed for projects using `Microsoft.Testing.Platform` (MTP) as the test runner, not the traditional VSTest runner.
### Usage (coverlet.MTP)
Coverlet integrates with the Microsoft Testing Platform as an extension. To enable coverage collection, run your tests with the `--coverlet` flag:
```bash
dotnet run --project <your-test-project> -- --coverlet
```
Or when using `dotnet test` with MTP-enabled projects:
```bash
dotnet test --coverlet
```
After the above command is run, coverage report files will be generated in the test results directory. By default, reports are generated in `json` and `cobertura` formats.
#### Additional Options (coverlet.MTP)
| Option | Description |
| -------- | ------------- |
| `--coverlet` | Enable code coverage data collection |
| `--coverlet-output-format` | Output format(s) for coverage report (json, lcov, opencover, cobertura) |
| `--coverlet-include` | Include assemblies matching filters (e.g., [Assembly]Type) |
| `--coverlet-exclude` | Exclude assemblies matching filters (e.g., [Assembly]Type) |
| `--coverlet-include-test-assembly` | Include test assembly in coverage |
Example with options:
```bash
dotnet run --project <your-test-project> -- --coverlet --coverlet-output-format cobertura --coverlet-exclude "[xunit.]"
```
#### Requirements (coverlet.MTP)
* *.NET 8.0 SDK or newer*
* *Test project configured for Microsoft Testing Platform*
* *Reference to `Microsoft.Testing.Platform` packages*
### VSTest Integration (preferred due to [known issue](https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/KnownIssues.md#1-vstest-stops-process-execution-earlydotnet-test))
### Installation (coverlet.collector)
```bash
dotnet add package coverlet.collector
```
> [!NOTE]
> You **MUST** add package only to test projects and if you create xunit test projects (`dotnet new xunit`) you will find the reference already present in `csproj` file because Coverlet is the default coverage tool for every .NET Core and >= *.NET 8* applications, you've only to update to last version if needed. Add `coverlet.collector` *OR* `coverlet.msbuild` package in a test project.
### Usage (coverlet.collector)
Coverlet is integrated into the Visual Studio Test Platform as a [data collector](https://github.com/Microsoft/vstest-docs/blob/master/docs/extensions/datacollector.md). To get coverage simply run the following command:
```bash
dotnet test --collect:"XPlat Code Coverage"
```
After the above command is run, a `coverage.cobertura.xml` file containing the results will be published to the `TestResults` directory as an attachment.
See [documentation](Documentation/VSTestIntegration.md) for advanced usage.
#### Requirements (coverlet.collector)
* *You need to be running .NET 8.0 SDK v8.0.112 or newer*
* *You need to reference version 17.12.0 and above of Microsoft.NET.Test.Sdk*
```xml
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.12.0" />
```
### MSBuild Integration (suffers of possible [known issue](https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/KnownIssues.md#1-vstest-stops-process-execution-earlydotnet-test))
### Installation (coverlet.msbuild)
```bash
dotnet add package coverlet.msbuild
```
N.B. Typically you **MUST** add package only to test projects. Do not add `coverlet.msbuild` and `coverlet.collector` package in a test project.
### Usage (coverlet.msbuild)
Coverlet also integrates with the build system to run code coverage after tests. Enabling code coverage is as simple as setting the `CollectCoverage` property to `true`
```bash
dotnet test /p:CollectCoverage=true
```
After the above command is run, a `coverage.json` file containing the results will be generated in the root directory of the test project. A summary of the results will also be displayed in the terminal.
See [documentation](Documentation/MSBuildIntegration.md) for advanced usage.
#### Requirements (coverlet.msbuild)
Requires a runtime that support *.NET Standard 2.0 and above*
### .NET Global Tool ([guide](https://docs.microsoft.com/en-us/dotnet/core/tools/global-tools), suffers from possible [known issue](https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/KnownIssues.md#1-vstest-stops-process-execution-earlydotnet-test))
### Installation (coverlet.console)
```bash
dotnet tool install --global coverlet.console
```
### Usage (coverlet.console)
The `coverlet` tool is invoked by specifying the path to the assembly that contains the unit tests. You also need to specify the test runner and the arguments to pass to the test runner using the `--target` and `--targetargs` options respectively. The invocation of the test runner with the supplied arguments **must not** involve a recompilation of the unit test assembly or no coverage result will be generated.
The following example shows how to use the familiar `dotnet test` toolchain:
```bash
coverlet /path/to/test-assembly.dll --target "dotnet" --targetargs "test /path/to/test-project --no-build"
```
*Note: The `--no-build` flag is specified so that the `/path/to/test-assembly.dll` assembly isn't rebuilt*
See [documentation](Documentation/GlobalTool.md) for advanced usage.
#### Requirements (coverlet.console)
.NET global tools rely on a .NET Core runtime installed on your machine <https://docs.microsoft.com/en-us/dotnet/core/tools/global-tools#what-could-go-wrong>
.NET Coverlet global tool requires *.NET 8.0 or above*
## How It Works
Coverlet generates code coverage information by going through the following process:
### Before Tests Run
* Locates the unit test assembly and selects all the referenced assemblies that have PDBs.
* Instruments the selected assemblies by inserting code to record sequence point hits to a temporary file.
### After Tests Run
* Restore the original non-instrumented assembly files.
* Read the recorded hits information from the temporary file.
* Generate the coverage result from the hits information and write it to a file.
## Deterministic build support
Coverlet supports coverage for deterministic builds. The solution at the moment is not optimal and need a workaround.
Take a look at [documentation](Documentation/DeterministicBuild.md).
## Are you in trouble with some feature? Check on [examples](Documentation/Examples.md)
## Known Issues
Unfortunately we have some [known issues](Documentation/KnownIssues.md)
## Cake Add-In
If you're using [Cake Build](https://cakebuild.net) for your build script you can use the [Cake.Coverlet](https://github.com/Romanx/Cake.Coverlet) add-in to provide you extensions to dotnet test for passing Coverlet arguments in a strongly typed manner.
## Visual Studio Add-In
If you want to visualize coverlet output inside Visual Studio while you code, you can use the following addins depending on your platform.
### Windows
If you're using Visual Studio on Windows, you can use the [Fine Code Coverage](https://marketplace.visualstudio.com/items?itemName=FortuneNgwenya.FineCodeCoverage) extension.
Visualization is updated when you run unit tests inside Visual Studio.
### Mac OS
If you're using Visual Studio for Mac, you can use the [VSMac-CodeCoverage](https://github.com/ademanuele/VSMac-CodeCoverage) extension.
## Consume nightly build
We offer nightly build of master for all packages.
See the [documentation](Documentation/ConsumeNightlyBuild.md)
## Issues & Contributions
If you find a bug or have a feature request, please report them at this repository's issues section. See the [CONTRIBUTING GUIDE](CONTRIBUTING.md) for details on building and contributing to this project.
## Coverlet Team
Author and owner
* [Toni Solarin-Sodara](https://github.com/tonerdo) (Inactive)
Co-maintainers
* [David Müller](https://github.com/daveMueller)
* [Bert](https://github.com/Bertk)
* [Peter Liljenberg](https://github.com/petli) (Inactive)
* [Marco Rossignoli](https://github.com/MarcoRossignoli) (Inactive)
## Code of Conduct
This project has adopted the code of conduct defined by the Contributor Covenant
to clarify expected behavior in our community.
For more information, see the [.NET Foundation Code of Conduct](https://dotnetfoundation.org/code-of-conduct).
## Credits
Part of the code is based on work done by OpenCover team <https://github.com/OpenCover>
## License
This project is licensed under the MIT license. See the [LICENSE](LICENSE) file for more info.
## Supported by the [.NET Foundation](https://dotnetfoundation.org/)