Skip to content

Improve x-test-features diagnostics.#1676

Merged
BillyONeal merged 19 commits intomicrosoft:mainfrom
BillyONeal:feature-test-diagnostics
May 14, 2025
Merged

Improve x-test-features diagnostics.#1676
BillyONeal merged 19 commits intomicrosoft:mainfrom
BillyONeal:feature-test-diagnostics

Conversation

@BillyONeal
Copy link
Copy Markdown
Member

@BillyONeal BillyONeal commented May 9, 2025

Substantially overhauls how diagnostics are reported out of x-test-features:

  • Unexpected pass now lists where in the baseline a failure was marked.
  • Unexpected fails or cascades now suggest baseline edits based on the 'kind' of test being done.
  • Conflicting 'state's from baselines are now diagnosed.

Other changes:

  • "Options" which start with "core" now participate in the 'combined' feature test.
  • Reuse the 'missing dependencies' portion of ExtendedBuildResult rather than attempting to recompute this.
  • Use std::is_permutation when determining whether feature tests are the "same" rather than special casing sort then trying the N^2 version.
  • A lot of tests.

BillyONeal added 18 commits May 9, 2025 12:30
Looking to all calls of handle_feature_test_result:

429: cascade_reason is a join of install_plan.unsupported_features, and the if is guarded by that not being empty on 413.
447: cascade_reason is set to the install action display name
461: Actual isn't cascade.
477: Actual isn't cascade.
549: Actual isn't cascade.
557: It's not entirely clear to prove that failed_dependencies will be nonempty here. However, failed_dependencies is recalculating something already present in ExtendedBuildResult, unmet_dependencies, which is already closely associated with CascadedDueToMissingDependencies

Before:

test[core]:x64-windows resulted in the unexpected state cascade because the following dependencies did not build: vcpkg-fail-if-depended-upon:x64-windows

After:

test[core]:x64-windows was unexpectedly a cascading failure because the following dependencies are unavailable: vcpkg-fail-if-depended-upon[a]:x64-windows, vcpkg-fail-if-depended-upon[core]:x64-windows
…o avoid needing to name cascade_reason for non-cascade cases.

# Conflicts:
#	src/vcpkg/commands.test-features.cpp
…eparate' feature tests. Ensure that the syntax for combination-fails and feature-fails are correct. Always talk about feature names with the port name for clarity.
@BillyONeal BillyONeal force-pushed the feature-test-diagnostics branch from b82a1c9 to ed9ae01 Compare May 9, 2025 20:19
@BillyONeal BillyONeal marked this pull request as ready for review May 9, 2025 21:46
@BillyONeal BillyONeal merged commit 5c19c3b into microsoft:main May 14, 2025
7 checks passed
@BillyONeal BillyONeal deleted the feature-test-diagnostics branch May 14, 2025 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants