Inconsistency regarding installing from VCS

[marcokuchla]

• Pip version: 10.0.1
• Python version: 3.4.5 (conda env)
• Operating system: Ubuntu 18.04

Description:

I have a package (call it mypackageA) that is in a private git repository.

setup.py (package mypackageA)

...
setup_requires = ["setuptools_scm"]
...

pyproject.toml (mypackageA)

[build-system]
# Minimum requirements for the build system to execute.
requires = ["setuptools", "wheel", "setuptools_scm"]  # PEP 508 specifications.

I can install it directly using the following command:

pip install git+ssh://git@myrepo/myprojectA.git#egg=mypackageA==1.0.1

However, it does not honor the version requested:

Requested mypackageA==1.0.1 from git+ssh://git@myrepo/mypackageA.git#egg=mypackageA==1.0.1, but installing version 1.0.2

And when declaring the mypackageA as a dependency to another package (say it myprojectB), like this:

setup.py (package mypackageB)

...
install_requires = ['myprojectA @ git+ssh://git@myrepo/myprojectA.git@1.0.1']
...

pyproject.toml (mypackageB)

[build-system]
# Minimum requirements for the build system to execute.
requires = ["setuptools", "wheel", "setuptools_scm"]  # PEP 508 specifications.

And trying to install mypackageB:

pip install -e .

The command fails:

Direct url requirement (like mypackageA @ git+ssh://myrepo/mypackageA.git@1.0.1) are not allowed for dependencies

But using:

setup.py (package mypackageB)

...
dependency_links=['git+ssh://git@myrepo/myprojectA.git#egg=myprojectA-1.0.1']
install_requires = ['myprojectA==1.0.1']
...

Then running:

pip install -e . --process-dependency-links

The installation successfully occurs (despite the deprecation warning regarding '--process-dependency-links'), but it installs the last available version from the repository, ignoring the versions specified at "#egg" and "install_requires".

So my questions are:

1. What is the alternative to "dependency_links" property since it is being deprecated?
2. Is there a way to pin a version using "dependency_links"?

[pradyunsg]

pip install git+ssh://git@myrepo/myprojectA.git#egg=mypackageA==1.0.1

The #egg= fragment is not for specifying a PEP 508 specification.

What is the alternative to "dependency_links" property since it is being deprecated?

That's #4187.

Is there a way to pin a version using "dependency_links"?

I think if you use the @tag syntax, you'll be able to specify the version.

[pradyunsg]

If that addresses your concerns, please close this issue.

If not, feel free to ask more questions here. :)

[marcokuchla]

Thanks for answering @pradyunsg,

I've read #4187 and as far I can see, there is no definitive alternative yet. About using the @tag syntax, I guess it will work, however, this causes an implication that every time a library is updated, I will need to change the dependency url to another specific version:

git+ssh://git@myrepo/myprojectA.git@1.0.0#egg=mypackageA==1.0.0
git+ssh://git@myrepo/myprojectA.git@1.0.1#egg=mypackageA==1.0.1
...

I want to know if there is a way to use the "compatible" version definition, like:

myProjectA~=1.0

and then specify the url as:

git+ssh://git@myrepo/myprojectA.git
...

And let pip figures out with version to checkout from the repo.
It is not clear whether PEP 508 can handle this situation, but I assume it will since it doesn't require #fragment in the url definition.

[pradyunsg]

How pip deals with VCS repositories today is given a VCS repo, it checks out either the provided tag/revision or the default ref/revision (i.e. master on git repos?). This is used as a source distribution. So, really, pip only ever sees the repository in one state.

I don't think it's feasible for pip to look into the history of a git repository to determine whether there's a compatible version available. That would require either checking out each revision and processing it to get the version metadata or making assumptions about the tagging scheme etc.

---

I think one approach that might work for you here is to have a develop branch and a master branch. The master branch will only contain code that you would want pip to be fetching, develop would contain in-development versions. Whenever you'd want to make a "release", just merge develop into master. (you can call them master and release respectively instead; that's obviously up to you)

That way, your requirement would just be git+ssh://private.server/private-project.git@master.

[pradyunsg]

PS: @tag sorry for the random mentions. Hope you have a good day. :)

[marcokuchla]

Hum, I see. Thanks for answering.

I think one approach that might work for you here is to have a develop branch and a master branch. The master branch will only contain code that you would want pip to be fetching, develop would contain in-development versions. Whenever you'd want to make a "release", just merge develop into master.

This is exactly how I keep my projects, using the Git Workflow with semantic versioning. It works fine when releasing patches or minors (since pip always installs latest snapshot avaiable in a branch), but if I release a new major and some project, that depends on that library, brokes with the new update, how to limit the project to a "compatible" version (like >=3.0,<4)?

I understand that I could use the url git+ssh://private.server/private-project.git@[some-specific-version-tag], to pin to a specific version, but then there is no flexibility when patches and minors are released.

I've found this link that explains the use of a option --find-links to be able to use the compatible version requirement, but I couldn't test it yet.

Then my question is, should I try the link's approach or go for a private PyPI? What would be better?

[marcokuchla]

I will close the issue. I'm sticking with pinned versions using @tag syntax.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.