Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix incorrect name check when autoloading required modules #8218

Merged

Conversation

rjmholt
Copy link
Collaborator

@rjmholt rjmholt commented Nov 9, 2018

PR Summary

Fixes #8204.

The change I made in #7125 made the check too strict on the module we try to autoload it from requires and rejected it based on name.

This PR removes that name check but also sets up a fix for the more general problem that a ModuleSpecification can always specify a path rather than a name. I added some reusable logic to make this easier to service.

The main contribution is regression testing for RequiredModules, #requires -Modules and other cmdlets that can use ModuleSpecifications.

However, the general pervasive problem is that ModuleSpecifications contain raw user input and we don't do very much about handling paths that might be given as module names, namely:

  • Normalising directory separators
  • Resolving relative paths (which should be resolved differently depending on context)
  • Dealing with case-sensitivity
  • Handling trailing slashes
  • Matching on module directories accounting for the versioned directory structure
  • Working with special directory names like . and ..

The methods I've added deal with these and I've applied them in one place to help correct the immediate problem and make it more efficient.

But the fact remains that we have no way in the type system to identify when this resolution has already occurred; there are over 120 uses of the ModuleSpecification type in the code base, and we don't have a way to differentiate when they contain raw user input and when they can be trusted to have a canonical path. The way the module cmdlets are written currently, we redo a fair amount of this validation and normalisation logic, and it's probably somewhere we could find serious perf improvements.

PR Checklist

@bergmeister
Copy link
Contributor

Thanks for handling the issue. I'd consider a regression test for this important feature to be essential.

@rjmholt
Copy link
Collaborator Author

rjmholt commented Nov 9, 2018

Agreed that it's essential -- I'm pretty surprised we didn't have one already.

I'm halfway through writing some already.

@rjmholt rjmholt force-pushed the fix-required-module-load-failure branch from 3205e0d to ae5c56b Compare November 9, 2018 09:05
@bergmeister
Copy link
Contributor

Ok, feel free to add the test in a 2nd PR as I can understand that this is a time-pressing issue that can initially be verified with some manual testing (please include Windows as test platform though)

@rjmholt
Copy link
Collaborator Author

rjmholt commented Nov 9, 2018

We also seem to do required module autoloading in #requires pragmas:

ModuleCmdletBase.LoadRequiredModule(
context: context,
currentModule: null,
requiredModuleSpecification: requiredModule,
moduleManifestPath: null,
manifestProcessingFlags: ModuleCmdletBase.ManifestProcessingFlags.LoadElements | ModuleCmdletBase.ManifestProcessingFlags.WriteErrors,
error: out error);

If those are supposed to allow paths, I'll need to do the same path normalisation logic there.

@adityapatwardhan
Copy link
Member

@rjmholt 1 new test is failing on Linux and macOS. Please have a look.

@rjmholt
Copy link
Collaborator Author

rjmholt commented Nov 12, 2018

The failing AppVeyor test is passing on my machine. Looking into it.

// Paths in the module name may fail here because
// they the wrong directory separator or are relative.
// Fix with the code below:
// FullyQualifiedName = FullyQualifiedName.Select(ms => ms.WithNormalizedName(Context, SessionState.Path.CurrentLocation.Path)).ToArray();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the concern about putting this fix in place? Regression? Perf degradation?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just keeping the fix constrained in this PR. It shouldn't cause regressions or perf degradations, but to ensure there are no regressions (since this PR is a regression fix itself) I just left it as a comment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will you open issue to track all those todo comments? They can be easily forgotten :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you open an issue to track those todo's?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Next on my list

// Paths in the module name may fail here because
// they the wrong directory separator or are relative.
// Fix with the code below:
// FullyQualifiedName = FullyQualifiedName.Select(ms => ms.WithNormalizedName(Context, SessionState.Path.CurrentLocation.Path)).ToArray();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you open an issue to track those todo's?

Copy link
Member

@daxian-dbw daxian-dbw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only pending action item left is to open an issue tracking all "TODO" comments.

@adityapatwardhan
Copy link
Member

Since the CodeFactor issues are for existing methods, re-ordering will cause larger diffs. We can ignore CodeFactor issues for this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CL-Engine Indicates that a PR should be marked as an engine change in the Change Log
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RequiredModules manifest property stopped working recently (noticed in daily build)
5 participants