Check CompatiblePSEditions manifest field for modules loaded from Windows PowerShell $PSHOME#7183
Conversation
0286a3f to
c6b5c14
Compare
BrucePay
left a comment
There was a problem hiding this comment.
I think you should fix the scriptblock creates in the tests. (Unless I'm missing something and there's a real reason for doing it that way.)
There was a problem hiding this comment.
If the only thing it's checking is the edition, maybe it should be called SkipEditionCheck ? Hmm - but maybe it should override the version check too? Then compatibility makes sense.
There was a problem hiding this comment.
I agree! My concern is that SkipCompatibilityCheck is what is given in the RFC.
There was a problem hiding this comment.
I think it's fine to change it to SkipEditionCheck as it's more specific
There was a problem hiding this comment.
I'll update the RFC to reflect this change
There was a problem hiding this comment.
Why change this from var?
There was a problem hiding this comment.
I can change it back if desired, but my reasoning is:
- The return type of
GetAvailableViaCimSessionCore(...)isn't inferable with only that line, or even only that file - Reading the code without an IDE is something we have to do relatively often and is impaired by non-obvious
var - Being explicit about what's expected as a return type means:
- Compile-time errors from any changes occur on that line (I've seen type inference push them into functions and different files)
- There's a type-as-contract concept at work, which explicitly defines what's expected and promotes superclasses as a sort of "contract loosening"
But that is definitely all opinion! As I say, I can revert it if the previous style is preferred.
There was a problem hiding this comment.
Same comment as Get-Module
There was a problem hiding this comment.
Why not just do & "Test-$ModuleName" | Should -Be $Result
There was a problem hiding this comment.
Why not just do {Import-Module $ModuleName -ErrorAction Stop; & "Test-$ModuleName"} | Should -Throw -ErrorId "Modules_PSEditionNotSupported,Microsoft.PowerShell.Commands.ImportModuleCommand"
There was a problem hiding this comment.
Again & "Test-$ModuleName" | Should -Be $Result
There was a problem hiding this comment.
I think it's fine to change it to SkipEditionCheck as it's more specific
There was a problem hiding this comment.
It seems that the two are not mutually exclusive. -PSEdition is for filtering while -SkipCompatibilityCheck is for enforcement.
There was a problem hiding this comment.
These test modules are pretty basic. I wonder if it's better to just generate these at runtime as test variations?
There was a problem hiding this comment.
Seems overkill since it doesn't appear the tests need to push more than once before a pop. Existing tests simply store currentPSModulePath in Before and restore in After
There was a problem hiding this comment.
Perhaps else is more readable.
There was a problem hiding this comment.
Perhaps else is more readable.
There was a problem hiding this comment.
We use compatiblePSEditions ?? DefaultCompatiblePSEditions twice. Maybe use a variable?
There was a problem hiding this comment.
Should we add defaults too?
There was a problem hiding this comment.
My thinking is that the PSModuleInfo should reflect the manifest. But I think this particular point needs further discussion, because rewriting default logic everywhere is error prone. I'm reluctant to add another public field on to the PSModuleInfo (and that would probably merit an RFC anyway).
But so far, the way it works is:
PSModuleInfo.CompatiblePSEditionsis exactly what's in the manifest (or an empty array)- Everywhere we check
CompatiblePSEditionsa default is used if it's null or empty
There was a problem hiding this comment.
I meant that if compatiblePSEditions == null we should add default.
There was a problem hiding this comment.
Yeah, I'm thinking that we shouldn't.
If we do, then we are not reflecting what the manifest says. So there will be no way to differentiate between a manifest that has CompatiblePSEditions = @("Desktop") and no CompatiblePSEditions field at all -- all our tools to parse a manifest will incorrectly see them as the same.
For the purposes of this logic, yes we assume a default. But this is quite specific logic. So I think we need to preserve the information for other functionalities that may need to distinguish the scenarios.
But maybe we also need a way to record the default assumed value on the PSModuleInfo?
There was a problem hiding this comment.
I agree with @rjmholt that the member should reflect what is actually in the manifest.
There was a problem hiding this comment.
Can we use Path.Combine()?
There was a problem hiding this comment.
This is to do "C:\path\one;C:\path\two" + "C:\path\three"; we're not combining paths into one path, but concatenating multiple paths into the PSModulePath using a path separator (';'). Is there a method for that?
There was a problem hiding this comment.
No, sorry - I skipped this.
#Closed.
There was a problem hiding this comment.
Please remove the unneeded comment.
#Closed.
There was a problem hiding this comment.
We can place the comment in one line.
#Closed.
There was a problem hiding this comment.
We can place the comment in one line.
#Closed.
There was a problem hiding this comment.
Should we use imperative mood in Use 'Import-Module -SkipCompatibilityCheck'?
Also {0} can be long path.
SteveL-MSFT
left a comment
There was a problem hiding this comment.
The formatter for ModuleInfo needs to be updated to expose PSEditions column.
There was a problem hiding this comment.
We should use HasFlag() - now it is optimized.
There was a problem hiding this comment.
Oh didn't see this before! Good idea!
There was a problem hiding this comment.
I wonder that we move the block - have we I side effect in lines 3450-3451?
There was a problem hiding this comment.
I wonder that we move the block - have we I side effect in lines 3450-3451?
There was a problem hiding this comment.
The if block? My goal is for the PSModuleInfo to accurately reflect the module manifest being loaded. It seems to be working for our purposes at present -- I moved the block down because I realised isOnSystem32ModulePath wasn't always set when a module was actually found there.
There was a problem hiding this comment.
AppVeyor is failing this test, but it's passing on my machine. Trying the command itself gives me the right exception. Stepping through the code in the debugger does what I expect. Any ideas?
There was a problem hiding this comment.
Why we use SilentlyContinue not Stop?
There was a problem hiding this comment.
What is benefits from moving the check on up level?
There was a problem hiding this comment.
Please remove first space before All.
There was a problem hiding this comment.
The same. What is benefits from moving the check on up level?
There was a problem hiding this comment.
The callee assertion seems to be a common occurrence in the codebase -- I could have included a comment, but thought an assertion was better as "zero-cost executable documentation".
Putting the actual if-check inside the method means needing to have a separate for-loop to handle the unchecked enumeration (since we can't use both return and yield return in the same method). Which doubles the size of the method and I think makes it less readable too. Doing it like this, we can avoid the whole method call (and in the case of the condition not being met, an unneeded layer of indirection in the enumeration) ahead of time and keep the method itself cohesive.
There was a problem hiding this comment.
return -> yield break?
we can avoid the whole method call
I think a performance is not so important here.
There was a problem hiding this comment.
Well yield break will return an empty enumeration. But if PSEdition is not set you don't want that, you want everything. So you need another for-loop just for the case where it's not set, which seems pretty ugly to me compared to just opting out of the filter.
Yes I very much agree that the performance gain is minimal and not needed. Just pointing out a separate virtue of the approach I took.
There was a problem hiding this comment.
I think the # if UNIX part can be abstracted in IsPSEditionCompatible, and then code here don't need to be divided up.
The IsPSEditionCompatible would look like this:
private bool IsPSEditionCompatible(
string moduleManifestPath,
IEnumerable<string> compatiblePSEditions,
out bool isOnSystem32ModulePath)
{
isOnSystem32ModulePath = false;
#if UNIX
return false;
#else
string windowsPSModulePath = ModuleIntrinsics.GetWindowsPowerShellPSHomeModulePath();
if (!moduleManifestPath.StartsWith(windowsPSModulePath, StringComparison.OrdinalIgnoreCase))
{
return true;
}
isOnSystem32ModulePath = true;
return BaseSkipEditionCheck || Utils.IsPSEditionSupported(compatiblePSEditions);
#endif
}There was a problem hiding this comment.
Yes I thought I'd start with the "zero runtime overhead on UNIX" approach and see how everyone felt. I definitely think your way is cleaner!
There was a problem hiding this comment.
It's better to have a static field keeping the value, so we only need to calculate it once.
There was a problem hiding this comment.
That's what I wanted to do. I assume the environment variable can't change then?
There was a problem hiding this comment.
System.Environment.SystemDirectory doesn't use the environment variable. It calls the win32 API GetSystemDirectoryW. See https://msdn.microsoft.com/en-us/library/windows/desktop/ms724373(v=vs.85).aspx
There was a problem hiding this comment.
Oh didn't see this before! Good idea!
There was a problem hiding this comment.
The if block? My goal is for the PSModuleInfo to accurately reflect the module manifest being loaded. It seems to be working for our purposes at present -- I moved the block down because I realised isOnSystem32ModulePath wasn't always set when a module was actually found there.
There was a problem hiding this comment.
I quite dislike this -- not sure if there's a better way to do it?
There was a problem hiding this comment.
I see we have private IEnumerable<PSModuleInfo> GetModuleForRootedPaths(string[] modulePaths, bool all, bool refresh)
There was a problem hiding this comment.
Going to change this over to the Utils method and optimise that one slightly. Want to let @daxian-dbw finish his review first (GitHub has had a nasty habit of forcing me to rewrite comments when new commits are added recently...)
There was a problem hiding this comment.
Why we use IEnumerable? We could use List - seems it is more thriftily.
There was a problem hiding this comment.
Used IEnumerable here because:
- The interface type defines all the functionality we need without added constraints
- We don't require indexing or even size checking, just ordered enumeration
- The method is free to implement the collection as it wishes
But, perhaps there are benefits to using a List or at least an IList?
- Less type complexity (and binding to a concrete type not an issue with an internal API)?
- Provides strictness and ordering guarantees
There was a problem hiding this comment.
I see IEnumerable benefits:
- it allows to implement generic algorithms. Seems we don't need this.
- it allows to postpone the execution (and implement interruption of long excution). Seems we don't need this too.
I believe we get more simple (readable and debuggable) code with List or Collection.
There was a problem hiding this comment.
I see we have private IEnumerable<PSModuleInfo> GetModuleForRootedPaths(string[] modulePaths, bool all, bool refresh)
There was a problem hiding this comment.
The file name doesn't contain Module. Maybe ModuleCompatiblePSEditions.Tests.ps1?
|
@iSazonov I couldn't reply to one of your comments for some reason. The After thinking about that, I thought that But maybe there's a more direct way to instantiate a cmdlet in a runspace and fire a method? |
|
So the After stepping through it all with the debugger, I can't find any difference in the behaviour... |
|
@BrucePay + @SteveL-MSFT, can you check to see if the changes I've made meet your requests? |
|
@daxian-dbw Should I rebase onto master again to take in your refactoring? |
|
We can put off the rebase after the PR has been signed off. |
8f7a74f to
f923bfb
Compare
| if (SkipEditionCheck && !ListAvailable) | ||
| { | ||
| ErrorRecord error = new ErrorRecord( | ||
| new InvalidOperationException("-SkipEditionCheck can only be used with -ListAvailable"), |
There was a problem hiding this comment.
Exception message should be put in resource files.
| { | ||
| // If we're trying to load the module, return null so that caches | ||
| // are not polluted | ||
| if (manifestProcessingFlags.HasFlag(ManifestProcessingFlags.LoadElements)) |
There was a problem hiding this comment.
I believe you can use importingModule here, instead of checking the flag again.
|
|
||
| // Don't cache incompatible modules on the system32 module path even if loaded with | ||
| // -SkipEditionCheck, since it will break subsequent sessions. | ||
| if (!module.IsConsideredEditionCompatible && ModuleUtils.IsOnSystem32ModulePath(module.ModuleBase)) |
There was a problem hiding this comment.
Maybe the ModuleUtils.IsOnSystem32ModulePath(module.ModuleBase) part is not needed.
| /// True if the module is on the system32 module path (where edition compatibility is checked), false otherwise. | ||
| /// </param> | ||
| /// <returns>True if the module is compatible with the running PowerShell edition, false otherwise.</returns> | ||
| private bool IsPSEditionCompatible( |
There was a problem hiding this comment.
Maybe simpler to make this method check "if a module is considered compatible" (IsModuleConsideredCompatbile). The check of BaseSkipEditionCheck can be done at the place where we call this method.
There was a problem hiding this comment.
Possibly we can use the same method for AnalysisCache.ModuleIsEditionIncompatible by moving this method to ModuleUtils
| /// All sub-directories that could be a module folder will be searched. | ||
| /// </summary> | ||
| internal static IEnumerable<string> GetAllAvailableModuleFiles(string topDirectoryToCheck) | ||
| internal static IEnumerable<string> GetAllAvailableModuleFiles(string topDirectoryToCheck, bool skipEditionCheck) |
There was a problem hiding this comment.
Let's defer solving the Get-Module -List -All problem. We are not sure what is the right user experience yet. We can solve it as a bug fix.
There was a problem hiding this comment.
Please open tracking and reference issue.
There was a problem hiding this comment.
I think we agreed to not solve Get-Module -List -All within this PR, because we are not sure what's the right UX.
Anything changes since we last discussion?
| /// Modules loaded from the System32 module path will have this as true if they | ||
| /// have declared edition compatibility with PowerShell Core. | ||
| /// </summary> | ||
| internal bool IsConsideredEditionCompatible { get; set; } = true; |
There was a problem hiding this comment.
With this default value, directly importing a non-psd1 module from the system32 module path will succeed.
This might not be the desired behavior, but let's defer solving that problem.
3cdc8b1 to
d9b0f9d
Compare
| case "Remove-Module": | ||
| { | ||
| NativeCompletionModuleCommands(context, parameterName, /* loadedModulesOnly: */ true, /* isImportModule: */ false, result); | ||
| NativeCompletionModuleCommands(context, parameterName, result, loadedModulesOnly: true); |
There was a problem hiding this comment.
Named parameters look more readable (what is "result" here?).
There was a problem hiding this comment.
Very much agree with you here. Sadly the parameter name is also result.
| { | ||
| NativeCompletionModuleCommands(context, parameterName, /* loadedModulesOnly: */ false, /* isImportModule: */ true, result); | ||
| bool skipEditionCheck = boundArguments != null && boundArguments.ContainsKey("SkipEditionCheck"); | ||
| NativeCompletionModuleCommands(context, parameterName, result, isImportModule: true, skipEditionCheck: skipEditionCheck); |
There was a problem hiding this comment.
Named parameters look more readable (what is "result" here?).
|
|
||
| // If we return null with Get-Module, a fake module info will be created. Since | ||
| // we want to suppress output of the module, we need to do that here. | ||
| return new PSModuleInfo(moduleManifestPath, context: null, sessionState: null) |
There was a problem hiding this comment.
Please open tracking and reference issues.
| /// All sub-directories that could be a module folder will be searched. | ||
| /// </summary> | ||
| internal static IEnumerable<string> GetAllAvailableModuleFiles(string topDirectoryToCheck) | ||
| internal static IEnumerable<string> GetAllAvailableModuleFiles(string topDirectoryToCheck, bool skipEditionCheck) |
There was a problem hiding this comment.
Please open tracking and reference issue.
|
@daxian-dbw I think this should be ready to merge now |
|
|
||
| // A module is considered compatible if it's not on the System32 module path, or | ||
| // if it is and declared "Core" as a compatible PSEdition. | ||
| manifestInfo.IsConsideredEditionCompatible = isConsideredCompatible; |
There was a problem hiding this comment.
IsConsideredEditionCompatible should reflect the truth about whether a module is considered compatible, which shouldn't be affected by BaseSkipEditionCheck.
| /// All sub-directories that could be a module folder will be searched. | ||
| /// </summary> | ||
| internal static IEnumerable<string> GetAllAvailableModuleFiles(string topDirectoryToCheck) | ||
| internal static IEnumerable<string> GetAllAvailableModuleFiles(string topDirectoryToCheck, bool skipEditionCheck) |
There was a problem hiding this comment.
I think we agreed to not solve Get-Module -List -All within this PR, because we are not sure what's the right UX.
Anything changes since we last discussion?
| /// have declared edition compatibility with PowerShell Core. Currently, this field | ||
| /// is true for all non-psd1 module files, when it should not be. Being able to | ||
| /// load psm1/dll modules from the System32 module path without needing to skip | ||
| /// the edition check is considered a bug and should be fixed. |
There was a problem hiding this comment.
Please open an issue tracking this.
@iSazonov pointed out two places above that we also need issues to track.
…on SkipEditionCheck
|
@rjmholt Can you please respond to all open comments? |
daxian-dbw
left a comment
There was a problem hiding this comment.
LGTM. Will merge the PR once CI builds pass.
|
This feature PR was squash-merged because:
|
…'System32' module path (#7183) - Add `%WINDIR%\System32\WindowsPowerShell\v1.0\Modules` (Windows PowerShell $PSHOME) to the end of the default PSCore 6 module path (i.e. the module path as initially set at startup). - Cause an error to be thrown by `Import-Module` when a module with `CompatiblePSEditions` not containing `"Core"` is being loaded from the 'System32' module path. - Suppress output of modules listed by `Get-Module -ListAvailable` from Windows PowerShell $PSHOME when `CompatiblePSEditions` does not contain `"Core"`. - Introduce the `-SkipCompatibilityCheck` switch parameter on both `Import-Module` and `Get-Module` to respectively allow importing incompatible modules and listing incompatible modules. - Adds a `PSEdition` column to the `PSModuleInfo` table view format. - Ensures that completions are not given for incompatible modules on the System32 module path.
|
@rjmholt Great work! Thanks! |
PR Summary
Resolves #6992, #6991, #6990. Also see PowerShell/PowerShell-RFC#130.
%WINDIR%\System32\WindowsPowerShell\v1.0\Modules(Windows PowerShell $PSHOME) to the end of the default PSCore 6 module path (i.e. the module path as initially set at startup).Import-Modulewhen a module withCompatiblePSEditionsnot containing"Core"is being loaded from the 'System32' module path.Get-Module -ListAvailablefrom Windows PowerShell $PSHOME whenCompatiblePSEditionsdoes not contain"Core".-SkipCompatibilityCheckswitch parameter on bothImport-ModuleandGet-Moduleto respectively allow importing incompatible modules and listing incompatible modules.PSEditioncolumn to thePSModuleInfotable view format.PR Checklist
.h,.cpp,.cs,.ps1and.psm1files have the correct copyright headerWIP:to the beginning of the title and remove the prefix when the PR is ready.[feature]if the change is significant or affects feature tests