Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate legacy metadata in temporary directory #7978

Merged
merged 4 commits into from
Apr 10, 2020

Conversation

sbidoul
Copy link
Member

@sbidoul sbidoul commented Apr 4, 2020

Progresses #7555 by getting rid of pip-egg-info as mentioned in #7882 (comment).

Pip generates legacy metadata by calling setup.py egg_info. For editables, it does that in the source directory, else it does it in a pip-egg-info subdirectory of source dir.

To avoid polluting source dir with this pip-egg-info if and when we will build in place, use a temporary directory instead.

Question to reviewers: can one think if there was a good reason to generate that pip-egg-info inside source dir?

If this PR sounds reasonable, one can further think about always generating legacy metadata in a temporary directory, even in the editable case. Indeed it looks like setup.py develop always runs egg_info even if the directory is already present. This would vastly simplify this area of the code. [update] this simplification in now part of this PR

@sbidoul sbidoul added the skip news Does not need a NEWS file entry (eg: trivial changes) label Apr 4, 2020
@pradyunsg
Copy link
Member

Editable installs need the egg-info to be in the location that it's in for working properly IIRC.

Other than that, this all sounds and looks good to me in principle.

@sbidoul
Copy link
Member Author

sbidoul commented Apr 4, 2020

Editable installs need the egg-info to be in the location that it's in for working properly IIRC.

Sure, but does pip need to have generate_metadata() put it there since setup.py develop that is invoked afterwards will generate it anyway.

@pradyunsg
Copy link
Member

pradyunsg commented Apr 4, 2020

I'm pretty sure that setuptools doesn't see that, so it should be fine to put that in a temporary directory.

Before it was generated in a pip-egg-info
subdirectory of the source dir. This will avoid
polluting source dir when we build in place.
@sbidoul
Copy link
Member Author

sbidoul commented Apr 5, 2020

Ok, I added a commit with the full simplification of metadata_legacy.py, using a temporary directory in all cases.

def _find_egg_info(source_directory, is_editable):
# type: (str, bool) -> str
"""Find an .egg-info in `source_directory`, based on `is_editable`.
def _find_egg_info(source_directory):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def _find_egg_info(source_directory):
def _find_egg_info(directory):

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function body and docstring also need to match this change :)

Always generate legacy metadata in a
temporary directory, even in the editable
case. Generating it in the source directory
did not add value, because setup.py develop
always regenerates the .egg-info directory.
Copy link
Member

@xavfernandez xavfernandez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice simplification 👍

@sbidoul
Copy link
Member Author

sbidoul commented Apr 8, 2020

Thanks for the review @xavfernandez

Copy link
Member

@pradyunsg pradyunsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These simplifications make me feel happy about factoring this code out of the depths of InstallRequirement + friends. :)

@xavfernandez xavfernandez added this to the 20.1 milestone Apr 8, 2020
@pradyunsg pradyunsg merged commit 9cbe8fb into pypa:master Apr 10, 2020
@sbidoul sbidoul deleted the remove-pip-egg-info-sbi branch April 10, 2020 13:28
@sbidoul
Copy link
Member Author

sbidoul commented May 17, 2020

@RonnyPfannschmidt this is the PR that removed pip-egg-info. I see there is a hack.py in setuptools_scm that relies on that directory so... yeah. That particular hack is going to be ineffective now. I'm not sure in which case it was needed though.

@RonnyPfannschmidt
Copy link
Contributor

When neither the scm nor a pkg-info file are part of the external source tree, setuptools_scm has no way to access the version information

@sbidoul
Copy link
Member Author

sbidoul commented May 17, 2020

@RonnyPfannschmidt yep, I know the case with setup.py in subdirectory, but in my experience it never worked with pip install/wheel . and setuptools_scm. Would you have a reproducer? Which pip version are you using to reproduce (must be master, because <20.1 does not have this PR and 20.1 builds in tree so should not be affected)?

@RonnyPfannschmidt
Copy link
Contributor

I don't have a reproducer, i observed this breakage before your pr, and AFAIK out of tree builds are coming back due to issue

@sbidoul
Copy link
Member Author

sbidoul commented May 17, 2020

Ok, then if it's unrelated to this PR, the problem you mentioned on discuss is not new, and is part of the known issues with out of tree builds, as summarized in #7555.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 15, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
skip news Does not need a NEWS file entry (eg: trivial changes)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants