Removing support for git+git@ VCS requirement scheme #7554

sbidoul · 2020-01-04T11:33:13Z

What's the problem this feature will solve?

In addition to VCS requirements which have the form of a proper URL, pip also partially supports a git specific scheme of the form git+git@example.com:user/repo.git. As currently documented, this scheme is supported for editable installs only.

Supporting this scheme has a number of drawbacks:

This creates confusion when people try to use it without the --editable option (pip requires the "-e" option to install from a scheme that starts with "git+git@" #2994, Wrong return value of a Link.is_artifact for git+user@hostname editable packages #6293).
This also creates bugs, such as when adding the commit to the requirement, pip freeze generates requirements that pip cannot process (pip freeze generating format that's incorrectly parsed by pip install -r #2038).
This specific format is not a proper URL, making such requirement impossible to translate to PEP 440 direct URL references.
it would seem that support for this format is rather accidental in the pip code base (pip freeze generating format that's incorrectly parsed by pip install -r #2038 (comment), also illustrated by the fact that link.is_vcs returns False for such pseudo-URLs), and properly supporting it everywhere would be very complex.

This scheme is easily replaced by URLs such as

git+https://git@example.com/...
git+ssh://git@example.com/...
git+git://git@example.com/... (insecure)

Describe the solution you'd like

Deprecate support for this format during one year, then remove it (#7543).

Alternative solutions

fix issues as they are discovered (e.g. support this format for non-editable uses, make pip freeze transform the reference to a supported format)
do nothing, document the exceptions

The text was updated successfully, but these errors were encountered:

pradyunsg · 2020-01-23T20:51:01Z

This was released in 20.0, so 21.0 should remove this feature.

lericson · 2020-08-14T16:08:20Z

It is not possible to make a relative path with the URL syntax. This artificially limit the power of pip for no reason. git+… simply passes … to git clone …. For example, if I have a branch that I want to install as a dependency:

pip install -e git+./@mydep#egg=mydep

This WILL break existing software and there will be no fix.

lericson · 2020-08-14T16:26:11Z

Never mind, I found a workaround using the environment variable support in requirements.txt:

-e git+file://${PWD}@mydep#egg=mydep

sbidoul · 2020-08-14T16:41:16Z

@lericson interesting trick you found there.

Out of curiosity is this a real example using the current directory, and why can't you use -e .?

lericson · 2020-08-14T16:52:40Z

The idea is to bundle slightly modified dependencies as branches in Git, then let pip install them. I'm still trying to find out how well it works though.

lericson · 2020-08-14T16:59:54Z

Apart from needing a bit of an incantation to make the dependency remote refs into local refs, it works pretty good!

sbidoul · 2020-08-14T17:06:00Z

@lericson good to hear! Note that if you don't specifically need an editable install, you could also use the PEP 508 notation: mydep @ git+file://${PWD}@branch.

lericson · 2020-08-14T17:37:16Z

It actually works pretty well. The advantage of having an editable install is that it’s, well, editable. I do a fair amount of hacking on my dependencies if they’re not very mature projects. This fits that pattern well without complicating things for my users (and myself as a developer) - Ludvig

…

On 14 Aug 2020, at 19:06, Stéphane Bidoul ***@***.***> wrote: @lericson good to hear! Note that if you don't specifically need an editable install, you could also use the PEP 508 notation: mydep @ ***@***.*** — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

uranusjr · 2020-08-14T18:16:02Z

Be careful that PWD is not available on Windows, so you may need to jump through some hoops if you need to support that sometime.

lericson · 2020-08-15T14:54:10Z

Eh, I gave up on Windows compatibility a long time ago :-)

pfmoore · 2021-01-09T12:24:19Z

A quick note. Pip 21.0 is due for release in about 2 weeks (see #9282). This item is on the 21.0 milestone, so it either needs to be implemented and merged to master before then, or it will miss the release (and I'll move it to the 21.1 milestone).

sbidoul · 2021-01-09T13:58:30Z

I'm taking care of this for 21.0

sbidoul · 2021-01-10T11:09:04Z

Done in #9436

215: Update pip to 21.0.1 r=duckinator a=pyup-bot This PR updates [pip](https://pypi.org/project/pip) from **20.3.3** to **21.0.1**. <details> <summary>Changelog</summary> ### 21.0.1 ``` =================== Bug Fixes --------- - commands: debug: Use packaging.version.parse to compare between versions. (`9461 <https://github.com/pypa/pip/issues/9461>`_) - New resolver: Download and prepare a distribution only at the last possible moment to avoid unnecessary network access when the same version is already installed locally. (`9516 <https://github.com/pypa/pip/issues/9516>`_) Vendored Libraries ------------------ - Upgrade packaging to 20.9 ``` ### 21.0 ``` ================= Deprecations and Removals ------------------------- - Drop support for Python 2. (`6148 <https://github.com/pypa/pip/issues/6148>`_) - Remove support for legacy wheel cache entries that were created with pip versions older than 20.0. (`7502 <https://github.com/pypa/pip/issues/7502>`_) - Remove support for VCS pseudo URLs editable requirements. It was emitting deprecation warning since version 20.0. (`7554 <https://github.com/pypa/pip/issues/7554>`_) - Modernise the codebase after Python 2. (`8802 <https://github.com/pypa/pip/issues/8802>`_) - Drop support for Python 3.5. (`9189 <https://github.com/pypa/pip/issues/9189>`_) - Remove the VCS export feature that was used only with editable VCS requirements and had correctness issues. (`9338 <https://github.com/pypa/pip/issues/9338>`_) Features -------- - Add ``--ignore-requires-python`` support to pip download. (`1884 <https://github.com/pypa/pip/issues/1884>`_) - New resolver: Error message shown when a wheel contains inconsistent metadata is made more helpful by including both values from the file name and internal metadata. (`9186 <https://github.com/pypa/pip/issues/9186>`_) Bug Fixes --------- - Fix a regression that made ``pip wheel`` do a VCS export instead of a VCS clone for editable requirements. This broke VCS requirements that need the VCS information to build correctly. (`9273 <https://github.com/pypa/pip/issues/9273>`_) - Fix ``pip download`` of editable VCS requirements that need VCS information to build correctly. (`9337 <https://github.com/pypa/pip/issues/9337>`_) Vendored Libraries ------------------ - Upgrade msgpack to 1.0.2. - Upgrade requests to 2.25.1. Improved Documentation ---------------------- - Render the unreleased pip version change notes on the news page in docs. (`9172 <https://github.com/pypa/pip/issues/9172>`_) - Fix broken email link in docs feedback banners. (`9343 <https://github.com/pypa/pip/issues/9343>`_) .. note You should *NOT* be adding new change log entries to this file, this file is managed by towncrier. You *may* edit previous change logs to fix problems like typo corrections or such. To add a new change log entry, please see https://pip.pypa.io/en/latest/development/contributing/#news-entries .. towncrier release notes start ``` ### 20.3.4 ``` =================== Features -------- - ``pip wheel`` now verifies the built wheel contains valid metadata, and can be installed by a subsequent ``pip install``. This can be disabled with ``--no-verify``. (`9206 <https://github.com/pypa/pip/issues/9206>`_) - Improve presentation of XMLRPC errors in pip search. (`9315 <https://github.com/pypa/pip/issues/9315>`_) Bug Fixes --------- - Fixed hanging VCS subprocess calls when the VCS outputs a large amount of data on stderr. Restored logging of VCS errors that was inadvertently removed in pip 20.2. (`8876 <https://github.com/pypa/pip/issues/8876>`_) - Fix error when an existing incompatibility is unable to be applied to a backtracked state. (`9180 <https://github.com/pypa/pip/issues/9180>`_) - New resolver: Discard a faulty distribution, instead of quitting outright. This implementation is taken from 20.2.2, with a fix that always makes the resolver iterate through candidates from indexes lazily, to avoid downloading candidates we do not need. (`9203 <https://github.com/pypa/pip/issues/9203>`_) - New resolver: Discard a source distribution if it fails to generate metadata, instead of quitting outright. This implementation is taken from 20.2.2, with a fix that always makes the resolver iterate through candidates from indexes lazily, to avoid downloading candidates we do not need. (`9246 <https://github.com/pypa/pip/issues/9246>`_) Vendored Libraries ------------------ - Upgrade resolvelib to 0.5.4. ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pip - Changelog: https://pyup.io/changelogs/pip/ - Homepage: https://pip.pypa.io/ </details> Co-authored-by: pyup-bot <github-bot@pyup.io>

bwoodsend · 2021-04-20T16:25:16Z

The git+git@ style is still what is given by pip freeze. Surely that's not intentional?

Edit: Ahh, I see #9625 which answers my question. Ignore me...

triage-new-issues bot added the S: needs triage Issues/PRs that need to be triaged label Jan 4, 2020

sbidoul added C: vcs pip's interaction with version control systems like git, svn and bzr type: deprecation Related to deprecation / removal. labels Jan 4, 2020

triage-new-issues bot removed the S: needs triage Issues/PRs that need to be triaged label Jan 4, 2020

sbidoul added this to the 21.0 milestone Jan 4, 2020

mfickett mentioned this issue Apr 30, 2020

pip install -e does not recognise a conda environment as a virtualenv conda/conda#5861

Closed

uranusjr mentioned this issue Aug 31, 2020

Proposal: pip-specific environment variable expansion in requirements and constraints files #8828

Open

ivberg mentioned this issue Oct 11, 2020

Hydra dependency will not install - git@github.com:akhti/hydra.git facebookresearch/rebel#5

Closed

DaHuoKolmostar mentioned this issue Jan 7, 2021

Can we use GitHub deploy keys to get dependencies from multiple private repositories webfactory/ssh-agent#30

Closed

This comment has been minimized.

Sign in to view

uranusjr modified the milestones: 21.0, 21.1 Jan 9, 2021

This comment has been minimized.

Sign in to view

uranusjr removed this from the 21.1 milestone Jan 9, 2021

uranusjr added this to the 21.0 milestone Jan 9, 2021

sbidoul self-assigned this Jan 9, 2021

sbidoul mentioned this issue Jan 9, 2021

Remove deprecated support for VCS pseudo URLs #9436

Merged

sbidoul closed this as completed in #9436 Jan 19, 2021

martey mentioned this issue Feb 1, 2021

Pip install fails from master jazzband/django-flatblocks#33

Closed

wslulciuc mentioned this issue Feb 10, 2021

Use remote.origin.url on airflow integration test MarquezProject/marquez#939

Merged

jcugat mentioned this issue Feb 19, 2021

'pip freeze' generates VCS pseudo URL a la git+git@ #9625

Closed

1 task

bwoodsend mentioned this issue Apr 21, 2021

Fix pip freeze to use modern format for git repos #9822

Merged

github-actions bot locked as resolved and limited conversation to collaborators Sep 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Removing support for git+git@ VCS requirement scheme #7554

Removing support for git+git@ VCS requirement scheme #7554

sbidoul commented Jan 4, 2020 •

edited

Loading

pradyunsg commented Jan 23, 2020

lericson commented Aug 14, 2020 •

edited

Loading

lericson commented Aug 14, 2020

sbidoul commented Aug 14, 2020

lericson commented Aug 14, 2020

lericson commented Aug 14, 2020

sbidoul commented Aug 14, 2020

lericson commented Aug 14, 2020 via email

uranusjr commented Aug 14, 2020

lericson commented Aug 15, 2020

pfmoore commented Jan 9, 2021

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

sbidoul commented Jan 9, 2021

sbidoul commented Jan 10, 2021

bwoodsend commented Apr 20, 2021 •

edited

Loading

Removing support for git+git@ VCS requirement scheme #7554

Removing support for git+git@ VCS requirement scheme #7554

Comments

sbidoul commented Jan 4, 2020 • edited Loading

pradyunsg commented Jan 23, 2020

lericson commented Aug 14, 2020 • edited Loading

lericson commented Aug 14, 2020

sbidoul commented Aug 14, 2020

lericson commented Aug 14, 2020

lericson commented Aug 14, 2020

sbidoul commented Aug 14, 2020

lericson commented Aug 14, 2020 via email

uranusjr commented Aug 14, 2020

lericson commented Aug 15, 2020

pfmoore commented Jan 9, 2021

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

sbidoul commented Jan 9, 2021

sbidoul commented Jan 10, 2021

bwoodsend commented Apr 20, 2021 • edited Loading

sbidoul commented Jan 4, 2020 •

edited

Loading

lericson commented Aug 14, 2020 •

edited

Loading

bwoodsend commented Apr 20, 2021 •

edited

Loading