Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop installing setuptools and wheel on Python 3.12+ #218

Merged
merged 1 commit into from
Aug 26, 2024

Conversation

edmorley
Copy link
Contributor

@edmorley edmorley commented Jul 7, 2024

Currently get-pip.py installs not only pip, but also setuptools and wheel by default, unless the --no-setuptools / --no-wheel (or PIP_NO_SETUPTOOLS / PIP_NO_WHEEL env vars) are used.

This has historically been necessary, however, modern versions of pip will now fallback to pyproject.toml (PEP 517) based builds (which will default to a setuptools backend, and thus automatically install setuptools and wheel in the isolated build environment) if either setuptools is not installed (as of pip 22.1), or if wheel is not installed (as of pip 23.1).

In addition, as of Python 3.12, the stdlib's ensurepip and venv modules no longer install setuptools, and only install pip.

As such, it is now time for get-pip.py to stop installing setuptools and wheel by default on Python 3.12+, in order to:

  • Improve parity with the behaviour of ensurepip / venv on Python 3.12+.
  • Act as another small step towards pyproject.toml / PEP 517 based builds eventually becoming the pip default. (Cross-reference: Default to isolated, pyproject.toml-based builds pip#9175)
  • Allow get-pip.py to focus on its primary responsibility: bootstrapping Pip.

Users can still request that setuptools and/or wheel are installed on Python 3.12+, by adding them to the get-pip.py invocation like so:

$ python get-pip.py setuptools wheel

Closes #200.

@edmorley
Copy link
Contributor Author

Please could I have review? :-)

README.md Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
templates/default.py Outdated Show resolved Hide resolved
@pradyunsg
Copy link
Member

Please could I have review? :-)

Yes, now that I have some bandwidth.

Currently `get-pip.py` installs not only pip, but also setuptools and
wheel by default, unless the `--no-setuptools` / `--no-wheel` (or
`PIP_NO_SETUPTOOLS` / `PIP_NO_WHEEL` env vars) are used.

This has historically been necessary, however, modern versions of pip
will now fallback to `pyproject.toml` (PEP 517: [1]) based builds (which
will default to a setuptools backend, and thus automatically install
setuptools and wheel in the isolated build environment) if either
setuptools is not installed (as of pip 22.1: [2]), or if wheel is not
installed (as of pip 23.1: [3]).

In addition, as of Python 3.12, the stdlib's `ensurepip` and `venv`
modules no longer install setuptools, and only install pip ([4]).

As such, it is now time for `get-pip.py` to stop installing setuptools
and wheel by default on Python 3.12+, in order to:
- Act as another small step towards `pyproject.toml` / PEP 517
  based builds eventually becoming the pip default.
- Improve parity with the behaviour of `ensurepip` / `venv` on
  Python 3.12+.
- Allow `get-pip.py` to focus on its primary responsibility:
  bootstrapping Pip.

Closes pypa#200.

[1]: https://peps.python.org/pep-0517/
[2]: pypa/pip#10717
[3]: pypa/pip#11871
[4]: python/cpython#101039
@edmorley
Copy link
Contributor Author

@pradyunsg Thank you for the review! I've made those changes. I wouldn't normally force push, but since this repo appears to exclusively use standard merges rather than squash and merge, I wanted to avoid a "Apply review feedback" type commit from ending up in the log on main. To see the diff between the last version and this, use:
https://github.com/pypa/get-pip/compare/2f463eb7e795d395a72f690eafca41b1b1fa8ea4..def4aec84b261b939137dd1c69eff0aabb4a7bf4

@edmorley edmorley requested a review from pradyunsg July 29, 2024 20:25
@edmorley
Copy link
Contributor Author

Testing with Python 3.12:

$ docker run --rm -it python:3.12 bash
...
root@e4b186d6ffd7:/# cd tmp/
root@e4b186d6ffd7:/tmp# python -m venv .venv --without-pip
root@e4b186d6ffd7:/tmp# source .venv/bin/activate
(.venv) root@e4b186d6ffd7:/tmp# curl -fLO https://github.com/edmorley/get-pip/raw/no-setuptools-wheel/public/get-pip.py
(.venv) root@e4b186d6ffd7:/tmp# python get-pip.py
Collecting pip
  Downloading pip-24.2-py3-none-any.whl.metadata (3.6 kB)
Downloading pip-24.2-py3-none-any.whl (1.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 17.0 MB/s eta 0:00:00
Installing collected packages: pip
Successfully installed pip-24.2

And with Python 3.11:

$ docker run --rm -it python:3.11 bash
...
root@6f0dbd95ed50:/# cd tmp/
root@6f0dbd95ed50:/tmp# python -m venv .venv --without-pip
root@6f0dbd95ed50:/tmp# source .venv/bin/activate
(.venv) root@6f0dbd95ed50:/tmp# curl -fLO https://github.com/edmorley/get-pip/raw/no-setuptools-wheel/public/get-pip.py
(.venv) root@6f0dbd95ed50:/tmp# python get-pip.py
Collecting pip
  Downloading pip-24.2-py3-none-any.whl.metadata (3.6 kB)
Collecting setuptools
  Downloading setuptools-72.1.0-py3-none-any.whl.metadata (6.6 kB)
Collecting wheel
  Downloading wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB)
Downloading pip-24.2-py3-none-any.whl (1.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 57.8 MB/s eta 0:00:00
Downloading setuptools-72.1.0-py3-none-any.whl (2.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 70.9 MB/s eta 0:00:00
Downloading wheel-0.43.0-py3-none-any.whl (65 kB)
Installing collected packages: wheel, setuptools, pip
Successfully installed pip-24.2 setuptools-72.1.0 wheel-0.43.0

edmorley added a commit to heroku/buildpacks-python that referenced this pull request Aug 7, 2024
Currently the buildpack performs a system site-packages install of not
only pip, but also setuptools and wheel. This has historically been
necessary for pip to be able to build source distributions (sdists) for
packages that don't ship with compatible wheels.

However:
- Thanks to PEP 518, packages can now (and many already do) specify an
  explicit build backend using `[build-system]` in their
  `pyproject.toml`. The dependencies specified in that config (such as
  setuptools and wheel) will be installed by pip into an isolated and
  ephemeral build environment as part of the source distribution build
  process. Such packages therefore don't need/use globally installed
  setuptools/wheel versions.
- As of pip v22.1, pip will now default to the isolated build environment
  mode (along with a fallback legacy setuptools build backend), if the
  setuptools package isn't installed globally. This means that packages
  that haven't yet migrated to a PEP 518 `pyproject.toml` build backend
  config can still build even if setuptools isn't installed globally.

There are a small number of rarely used packages in the wild that
aren't compatible with build isolation mode, however, these typically
require more build dependencies than just setuptools, which means they
wouldn't have worked with this buildpack anyway.

As such, it's no longer necessary for us to install setuptools and wheel
globally. This matches the behaviour of the `venv` and `ensurepip` modules
in Python 3.12+, where setuptools and wheel installation has also been
removed. And it also matches the default behaviour of Poetry too, whose
`install --sync` command removes any implicitly installed packages in the
current environment (other than pip).

See:
https://peps.python.org/pep-0518/
https://pip.pypa.io/en/stable/reference/build-system/
https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation
pypa/pip#10717
python/cpython#101039
pypa/get-pip#218
astral-sh/uv#2252

GUS-W-16437776.
edmorley added a commit to heroku/buildpacks-python that referenced this pull request Aug 7, 2024
Currently the buildpack performs a system site-packages install of not
only pip, but also setuptools and wheel. This has historically been
necessary for pip to be able to build source distributions (sdists) for
packages that don't ship with compatible wheels.

However:
- Thanks to PEP 518, packages can now (and many already do) specify an
  explicit build backend using `[build-system]` in their
  `pyproject.toml`. The dependencies specified in that config (such as
  setuptools and wheel) will be installed by pip into an isolated and
  ephemeral build environment as part of the source distribution build
  process. Such packages therefore don't need/use globally installed
  setuptools/wheel versions.
- As of pip v22.1, pip will now default to the isolated build environment
  mode (along with a fallback legacy setuptools build backend), if the
  setuptools package isn't installed globally. This means that packages
  that haven't yet migrated to a PEP 518 `pyproject.toml` build backend
  config can still build even if setuptools isn't installed globally.

There are a small number of rarely used packages in the wild that
aren't compatible with build isolation mode, however, these typically
require more build dependencies than just setuptools, which means they
wouldn't have worked with this buildpack anyway.

As such, it's no longer necessary for us to install setuptools and wheel
globally. This matches the behaviour of the `venv` and `ensurepip` modules
in Python 3.12+, where setuptools and wheel installation has also been
removed. And it also matches the default behaviour of Poetry too, whose
`install --sync` command removes any implicitly installed packages in the
current environment (other than pip).

See:
https://peps.python.org/pep-0518/
https://pip.pypa.io/en/stable/reference/build-system/
https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation
pypa/pip#10717
python/cpython#101039
pypa/get-pip#218
astral-sh/uv#2252

GUS-W-16437776.
edmorley added a commit to heroku/buildpacks-python that referenced this pull request Aug 7, 2024
Currently the buildpack performs a system site-packages install of not
only pip, but also setuptools and wheel. This has historically been
necessary for pip to be able to build source distributions (sdists) for
packages that don't ship with compatible wheels.

However:
- Thanks to PEP 518, packages can now (and many already do) specify an
  explicit build backend using `[build-system]` in their
  `pyproject.toml`. The dependencies specified in that config (such as
  setuptools and wheel) will be installed by pip into an isolated and
  ephemeral build environment as part of the source distribution build
  process. Such packages therefore don't need/use globally installed
  setuptools/wheel versions.
- As of pip v22.1, pip will now default to the isolated build environment
  mode (along with a fallback legacy setuptools build backend), if the
  setuptools package isn't installed globally. This means that packages
  that haven't yet migrated to a PEP 518 `pyproject.toml` build backend
  config can still build even if setuptools isn't installed globally.

There are a small number of rarely used packages in the wild that
aren't compatible with build isolation mode, however, these typically
require more build dependencies than just setuptools, which means they
wouldn't have worked with this buildpack anyway.

As such, it's no longer necessary for us to install setuptools and wheel
globally. This matches the behaviour of the `venv` and `ensurepip` modules
in Python 3.12+, where setuptools and wheel installation has also been
removed. And it also matches the default behaviour of Poetry too, whose
`install --sync` command removes any implicitly installed packages in the
current environment (other than pip).

See:
https://peps.python.org/pep-0518/
https://pip.pypa.io/en/stable/reference/build-system/
https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation
pypa/pip#10717
python/cpython#101039
pypa/get-pip#218
astral-sh/uv#2252

GUS-W-16437776.
@edmorley
Copy link
Contributor Author

Please could I have a re-review? :-)

@edmorley
Copy link
Contributor Author

Please may someone merge this, now that it's had review (and time for anyone else to comment/review/object)? :-)

@pradyunsg
Copy link
Member

Thanks for the poke @edmorley! I've merged this but we can revert this if someone raises concerns around this.

@edmorley edmorley deleted the no-setuptools-wheel branch August 26, 2024 18:03
@edmorley
Copy link
Contributor Author

Thank you :-)

As a reminder to anyone coming across this PR who wants to return to the old behaviour, you can update your get-pip.py usage to this to continue to install setuptools and wheel on Python 3.12+:

$ python get-pip.py setuptools wheel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stop installing setuptools and wheel by default
2 participants