[release/v7.4] Add GitHub Actions annotations for Pester test failures#26800
Conversation
…6789) Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: TravisEz13 <[email protected]> Co-authored-by: Travis Plunk <[email protected]>
TravisEz13
added
the
CL-Test
There was a problem hiding this comment.
Pull request overview
This pull request backports #26789 to the release/v7.4 branch, adding GitHub Actions workflow annotations for Pester test failures. When tests fail in GitHub Actions, the changes automatically generate clickable file annotations with direct links to the failing test code and workflow logs, improving debugging efficiency for CI/CD workflows.
Changes:
- Adds
Get-PesterFailureFileInfohelper function to parse Pester stack traces and extract file paths and line numbers - Extends
process-pester-results.ps1to generate GitHub Actions workflow command annotations for test failures - Supports both Pester 4 and Pester 5 stack trace formats with cross-platform path handling (Unix and Windows)
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| build.psm1 | Adds Get-PesterFailureFileInfo function to parse Pester stack traces and extract file location information using multiple regex patterns for Pester 4/5 compatibility |
| .github/actions/test/process-pester-results/process-pester-results.ps1 | Generates GitHub Actions annotations by parsing test failure XML, extracting file locations, converting absolute paths to relative paths, and outputting workflow commands with clickable links |
| $annotationTitle = "Test Failure: $description / $testName" | ||
|
|
||
| # Build the annotation message | ||
| $annotationMessage = $message -replace "`n", "%0A" -replace "`r" | ||
|
|
||
| # Build and output the workflow command | ||
| $workflowCommand = "::error file=$filePath" | ||
| if ($fileInfo.Line) { | ||
| $workflowCommand += ",line=$($fileInfo.Line)" | ||
| } | ||
| $workflowCommand += ",title=$annotationTitle::$annotationMessage" |
There was a problem hiding this comment.
The annotation title could potentially contain special characters like double colons (::) or commas (,) that have special meaning in GitHub Actions workflow commands. If a test name or description contains these characters, it could break the workflow command parsing.
Consider escaping or sanitizing the annotation title to prevent parsing issues. For example, replace : with a safe character or URL-encode special characters. This is particularly important if test names can contain arbitrary user input.
| # GitHub Actions expects paths relative to the workspace root | ||
| if ($env:GITHUB_WORKSPACE) { | ||
| $workspacePath = $env:GITHUB_WORKSPACE | ||
| if ($filePath.StartsWith($workspacePath)) { |
There was a problem hiding this comment.
The path comparison using .StartsWith($workspacePath) without specifying a StringComparison parameter is case-sensitive on Unix-like systems. This could cause the path conversion to fail on case-sensitive filesystems if the case doesn't match exactly.
For cross-platform compatibility, use .StartsWith($workspacePath, [System.StringComparison]::OrdinalIgnoreCase) to ensure case-insensitive comparison. This pattern is consistently used elsewhere in the codebase for path comparisons.
| if ($filePath.StartsWith($workspacePath)) { | |
| if ($filePath.StartsWith($workspacePath, [System.StringComparison]::OrdinalIgnoreCase)) { |
|
@copilot open a new pull request to apply changes based on the comments in this thread |
Backport of #26789 to release/v7.4
Triggered by @TravisEz13 on behalf of @app/copilot-swe-agent
Original CL Label: CL-Test
/cc @PowerShell/powershell-maintainers
Impact
REQUIRED: Choose either Tooling Impact or Customer Impact (or both). At least one checkbox must be selected.
Tooling Impact
Adds GitHub Actions workflow annotations for Pester test failures, providing clickable file annotations with direct links to failing test code and workflow logs. This improves debugging experience in CI but is not required for builds to succeed.
Customer Impact
Regression
REQUIRED: Check exactly one box.
This is not a regression.
Testing
Comprehensive testing performed including Pester 4/5 support, cross-platform path handling, absolute to relative path conversion, and PSScriptAnalyzer compliance. The feature was validated through actual CI execution in the original PR.
Risk
REQUIRED: Check exactly one box.
Medium risk - adds new functionality to CI test result processing without affecting core engine or existing code paths. Changes are well-contained to test infrastructure (build.psm1 and process-pester-results.ps1). The feature is backward compatible and has been validated in the original PR merge.