Skip to content

Commit 5cc27e2

Browse files
rjmholtadityapatwardhan
rjmholt
authored and
adityapatwardhan
committed
Add code coverage report generation instructions (#6515)
* Add code coverage report generation instructions * Add Publish-PSTestTools in docs * Add references to coverage comparison commands
1 parent 1be0594 commit 5cc27e2

4 files changed

Lines changed: 173 additions & 0 deletions

File tree

‎.spelling‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,7 @@ WSMan
207207
wsmansessionoption.cs
208208
xUnit
209209
0-powershell-crossplatform
210+
ReportGenerator
210211
#endregion
211212

212213
#region ./tools/install-powershell.readme.md Overrides

‎docs/testing-guidelines/Images/CoverageReportFilter.PNG‎

594 KB
Loading

‎docs/testing-guidelines/Images/CoverageReportTop.PNG‎

440 KB
Loading

‎docs/testing-guidelines/getting-code-coverage.md‎

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
# Getting Code Coverage Analysis for PowerShell
2+
3+
**Note: Code coverage is currently only supported on Windows, since we use OpenCover.**
4+
5+
The PowerShell code base is configured to build with code coverage support using [OpenCover].
6+
7+
You can see the testing coverage of the current [`master`] branch build at any time at [codecov.io].
8+
9+
To run test coverage analysis of PowerShell on your own branch/machine,
10+
you will need to take the following steps
11+
(and be aware that running the code coverage analysis can take as long as 8 hours).
12+
13+
## Running tests with code coverage analysis
14+
15+
**First**: Open PowerShell in an **elevated** session.
16+
OpenCover needs elevated privileges to work.
17+
18+
Now, in PowerShell:
19+
20+
```powershell
21+
# Go to your PowerShell build directory root
22+
PS> Set-Location "C:\Path\to\powershell\build\dir"
23+
24+
# Import the PowerShell build module
25+
PS> Import-Module .\build.psm1
26+
27+
# Build PowerShell. You may need to add other flags here like
28+
# -ResGen or -Restore
29+
PS> Start-PSBuild -Configuration CodeCoverage -Clean -PsModuleRestore
30+
31+
# Now ensure Pester is installed
32+
PS> Restore-PSPester
33+
34+
# We also need to build the test executor
35+
PS> Publish-PSTestTools
36+
37+
# Import the OpenCover module
38+
PS> Import-Module $PWD\test\tools\OpenCover
39+
40+
# Install OpenCover to a temporary directory
41+
PS> Install-OpenCover -TargetDirectory $env:TEMP -Force
42+
43+
# Finally, run the tests with code coverage analysis.
44+
# If you want to run only the continuous integration tests,
45+
# add -CIOnly, which will take less time
46+
PS> Invoke-OpenCover -OutputLog coverage.xml -OpenCoverPath $env:TEMP\OpenCover
47+
```
48+
49+
## Examining the code coverage data
50+
51+
Once the code coverage test run is done, you'll want to examine the data:
52+
53+
```powershell
54+
# Collect the coverage data using Get-CodeCoverage from the OpenCover
55+
# module that was imported above. This operation is generally expensive
56+
# to compute, so worth storing in a variable
57+
PS> $coverageData = Get-CodeCoverage .\coverage.xml
58+
59+
# Take a look at a summary of the results
60+
PS> $coverageData.CoverageSummary
61+
62+
NumSequencePoints : 298237
63+
VisitedSequencePoints : 125949
64+
NumBranchPoints : 101477
65+
VisitedBranchPoints : 39389
66+
SequenceCoverage : 42.23
67+
BranchCoverage : 38.82
68+
MaxCyclomaticComplexity : 393
69+
MinCyclomaticComplexity : 1
70+
VisitedClasses : 1990
71+
NumClasses : 3187
72+
VisitedMethods : 15115
73+
NumMethods : 32517
74+
75+
# You can also view results by assembly
76+
PS> $coverageData.Assembly | Format-Table AssemblyName,Branch,Sequence
77+
78+
AssemblyName Branch Sequence
79+
------------ ------ --------
80+
pwsh 100 100
81+
Microsoft.PowerShell.ConsoleHost 21.58 23.32
82+
System.Management.Automation 41.22 45.01
83+
Microsoft.PowerShell.CoreCLR.Eventing 1.88 2.03
84+
Microsoft.PowerShell.Security 17.32 20.09
85+
Microsoft.PowerShell.Commands.Utility 20.14 21.39
86+
Microsoft.PowerShell.Commands.Management 43.05 43.39
87+
Microsoft.WSMan.Management 52.58 56.98
88+
Microsoft.WSMan.Runtime 80.95 80.33
89+
Microsoft.PowerShell.Commands.Diagnostics 0 0
90+
```
91+
92+
If you have made changes to tests or code
93+
and run a second code coverage run,
94+
you can also compare code coverage results:
95+
96+
```powershell
97+
PS> $cov1 = Get-CodeCoverage ./coverage1.xml
98+
PS> $cov2 = Get-CodeCoverage ./coverage2.xml
99+
PS> Compare-CodeCoverage -Run1 $cov1 -Run2 $cov2
100+
101+
AssemblyName Sequence SequenceDelta Branch BranchDelta
102+
------------ -------- ------------- ------ -----------
103+
Microsoft.PowerShell.Security 20.09 -30.12 17.32 -31.63
104+
Microsoft.PowerShell.Commands.Management 43.39 9.10 43.05 11.59
105+
System.Management.Automation 45.04 -10.63 41.23 -11.07
106+
Microsoft.PowerShell.Commands.Utility 21.39 -47.22 20.14 -46.47
107+
Microsoft.PowerShell.Commands.Diagnostics 0 -51.91 0 -48.62
108+
Microsoft.PowerShell.ConsoleHost 23.32 -22.28 21.58 -22.47
109+
pwsh 100 0.00 100 0.00
110+
Microsoft.WSMan.Management 57.73 48.23 53.02 43.22
111+
Microsoft.WSMan.Runtime 80.33 -19.67 80.95 -19.05
112+
Microsoft.PowerShell.CoreCLR.Eventing 2.03 -32.74 1.88 -26.01
113+
```
114+
115+
To get file-specific coverage data,
116+
you can use `Compare-FileCoverage`:
117+
118+
```powershell
119+
PS> Compare-FileCoverage -ReferenceCoverage $cov2 -DifferenceCoverage $cov1 -FileName LanguagePrimitives.cs
120+
121+
FileName ReferenceCoverage DifferenceCoverage CoverageDelta
122+
-------- ----------------- ------------------ -------------
123+
LanguagePrimitives.cs 53.68 69.03 15.34
124+
```
125+
126+
You can see more ways to use `Compare-CodeCoverage` and `Compare-FileCoverage`
127+
by running:
128+
129+
```powershell
130+
PS> Get-Help Compare-CodeCoverage -Full
131+
# Or
132+
PS> Get-Help Compare-FileCoverage -Full
133+
```
134+
135+
## Visualizing code coverage
136+
137+
For a more detailed, graphical representation of the code coverage results,
138+
you can use the ReportGenerator package.
139+
This generates an HTML report of the coverage from the XML file
140+
and will provide much more detail about the coverage analysis.
141+
The package is available on [NuGet],
142+
and you can install and run it as follows:
143+
144+
```powershell
145+
# Install ReportGenerator
146+
PS> Find-Package ReportGenerator `
147+
>> -ProviderName Nuget `
148+
>> -Source "https://nuget.org/api/v2" `
149+
>> | Install-Package -Scope CurrentUser
150+
151+
# Get the ReportGenerator executable path
152+
# Make sure use the appropriate version number in the path
153+
$ReportGenExe = "$HOME\AppData\Local\PackageManagement\NuGet\Packages\ReportGenerator.<version>\tools\ReportGenerator.exe"
154+
155+
# Run ReportGenerator
156+
& $ReportGenExe -report:coverage.xml -targetdir:C:\temp\Coverage
157+
158+
# Finally, open the report in your browser
159+
Invoke-Item C:\temp\Coverage\index.htm
160+
```
161+
162+
This should open a screen in the browser like this:
163+
![Coverage report browser page](Images/CoverageReportTop.png)
164+
165+
The main report, which is below the summary and risk hot spots, has
166+
a filter functionality as well (when "Enable Filtering" is clicked on):
167+
![Coverage report with filter on](Images/CoverageReportFilter.png)
168+
169+
[OpenCover]: https://github.com/OpenCover/opencover
170+
[codecov.io]: https://codecov.io
171+
[`master`]: https://github.com/PowerShell/PowerShell
172+
[NuGet]: https://nuget.org/packages/ReportGenerator

0 commit comments

Comments
 (0)