Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python: create venv's --without-pip #15792

Merged
merged 1 commit into from
Jul 31, 2023

Conversation

branchvincent
Copy link
Member

@branchvincent branchvincent commented Jul 30, 2023

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew style with your changes locally?
  • Have you successfully run brew typecheck with your changes locally?
  • Have you successfully run brew tests with your changes locally?

Currently, virtualenv_install_with_resources creates a venv with a full copy of pip. This is pretty wasteful, since we really only need it at build time to install resources. Instead, we can:

  1. Create a venv using --without-pip
  2. Use python@x.y's pip to install resources into this venv, using the --python option
  3. Tangentially, remove all shell activate scripts (which admittedly doesn't save much but aren't necessary)

As a result, this saves ~7MB of disk space per venv:

$ brew install black
...
==> Summary
🍺  /opt/homebrew/Cellar/black/23.7.0_1: 1,121 files, 13.5MB
$ brew reinstall -s black
...
==> Summary
🍺  /opt/homebrew/Cellar/black/23.7.0_1: 558 files, 6.6MB, built in 41 seconds

Note I don't foresee this causing any issues, but I've left an opt-out to be safe.

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me! Could you add a commented-out odeprecated for the opt-out so we can remove it if it's unused? Thanks!

@branchvincent
Copy link
Member Author

Done!

@MikeMcQuaid
Copy link
Member

Thanks again @branchvincent!

@MikeMcQuaid MikeMcQuaid merged commit d1f60ae into Homebrew:master Jul 31, 2023
24 checks passed
@branchvincent branchvincent deleted the venv-no-pip branch July 31, 2023 18:21
@cangussu
Copy link

I have a bunch of machines running different brew versions and a formula that uses virtualenv_create and relies on the fact that pip is available. What is the proper way to deal with this change? It broke my formula with new version of brew but if I add the extra parameter it will break on older versions.

@branchvincent
Copy link
Member Author

Sorry about that!

running different brew versions

may be a silly question, but is updating to the latest version not an option?

relies on the fact that pip is available

in what way? there may be a simple change you could make, like Homebrew/homebrew-core#138661 (comment)

@MikeMcQuaid
Copy link
Member

I have a bunch of machines running different brew versions and a formula that uses virtualenv_create and relies on the fact that pip is available.

@cangussu Which versions? Note we do not support running any Homebrew versions beyond the latest.

@bebound
Copy link

bebound commented Aug 17, 2023

This also removes setuptools. Some namespace packages may rely on it:

linuxbrew@794b47da48b2:/down$ az -v
Traceback (most recent call last):
  File "/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/runpy.py", line 187, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/runpy.py", line 110, in _get_module_details
    __import__(pkg_name)
  File "/home/linuxbrew/.linuxbrew/Cellar/azure-cli/2.51.0/libexec/lib/python3.10/site-packages/azure/__init__.py", line 1, in <module>
    __import__('pkg_resources').declare_namespace(__name__)
ModuleNotFoundError: No module named 'pkg_resources'

@branchvincent
Copy link
Member Author

Since setuptools is included with our python, you'll only run into trouble if you've disabled system_site_packages

Also note if you use setuptools at runtime, you really should add it to your requirements as Python 3.12 will no longer include it either python/cpython#95299 pypa/pip#12032

@cangussu
Copy link

cangussu commented Sep 6, 2023

@branchvincent I followed your suggestion to use python -m pip and it worked for our use case, tks!

in what way? there may be a simple change you could make, like Homebrew/homebrew-core#138661 (comment)

@github-actions github-actions bot added the outdated PR was locked due to age label Oct 7, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 7, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
outdated PR was locked due to age
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants