Note issues with Provides-Dist and Obsoletes-Dist

[ncoghlan]

The design of Provides-Dist and Obsoletes-Dist was lifted pretty
much directly from the design of Linux system package managers,
without accounting for the fact that Linux distro repos are collaboratively
maintained curated collections of packages with the ability to make
repository wide changes, while PyPI is an open publishing platform
with few practical restrictions other than size quotas and the default
first-come-first-served policy on claiming names.

pypa/packaging-problems#154

[pganssle]

I think it's good to make this distinction, but as I mentioned in the other thread, I don't think that these sorts of things are nearly as likely to cause malicious problems as they seem. If anything the fact that they aren't enforced would cause more problems.

Until a long-term solution is clear, I'd probably go with a less prescriptive and more descriptive phrasing, something like, "Currently tools like pip do not make any use of this information, and you should be aware that the information is not validated or vetted in any way."

[ncoghlan]

The fact pip will currently let one project overwrite another project's files seems completely orthogonal to this issue to me. The answer to that is to get pip to stop doing it unless you explicitly tell it that it's OK.

[ncoghlan]

(Note: I do think Conflicts-Dist is a good idea to explicitly prevent co-installation, since that's a project level assertion that "Co-installation with this other project won't work". Still orthogonal to why the installation tools have never implemented support for Provides-Dist or Obsoletes-Dist in any meaningful way though, which is that in the absence of active curation of the repository as a whole, it turns out that there really isn't much that installers can safely do with the information)

[pfmoore]

The fact pip will currently let one project overwrite another project's files seems completely orthogonal to this issue to me. The answer to that is to get pip to stop doing it unless you explicitly tell it that it's OK.

Is there a pip issue for this? It sounds to me like a reasonable feature request.

[pradyunsg]

Is there a pip issue for this? It sounds to me like a reasonable feature request.

Yep -- pypa/pip#4625.

[pganssle]

The fact pip will currently let one project overwrite another project's files seems completely orthogonal to this issue to me. The answer to that is to get pip to stop doing it unless you explicitly tell it that it's OK.

Well they're at least related by the fact that a worse form of abuse than is possible with Obsoletes-Dist is already possible, and it does not seem to be causing major problems at the moment. I think the jury is or should still be out on whether pip should do anything with Obsoletes-Dist (I argue in the other thread that it should be a special case of Conflicts-Dist), which is why I would argue for documenting that these are not currently used for anything rather than what they should be used for.

[ncoghlan]

I have no idea whether or not anyone is using them successfully in a private use scenario - they're perfectly fine in the context of a Linux-style curated repository, which is why I've worded the PR the way I have. (Private curated repositories are also one of the main available mechanisms for folks to defend against undocumented conflicts)

[pganssle]

@ncoghlan I don't have any objection to using them in private curated repositories, but I think "should only be used" is too strong, because I find the argument in favor of this is pretty weak. If Obsoletes-Dist were actually used to either uninstall conflicting packages or just block the installation of conflicting packages, I think that would be an improvement over the status quo. Even if the package-overwriting behavior of pip were fixed, I don't see it as any sort of security issue to interpret Obsoletes-Dist as a special case of Conflicts-Dist (I think this is basically how it is used in curated repositories anyway, though the uninstall-the-old-thing logic may be automatic).

I'd be happier replacing the stronger wording with, "These are not currently used by pip or PyPI, but have historically been included in case they would be useful for other package managers, particularly curated repositories."

[ncoghlan]

Ah, I see now - yeah, that makes sense.

eb8c260 switches from my original prescriptive wording to more matter-of-fact descriptive wording.

[pganssle]

@ncoghlan Thanks for putting up with my persnickettiness!

[ankostis]

It's been referred in this issue and others the Conflicts-Dist metadata, but i cannot find anything about it.

What is it going to be working?
Would it be preferable over Obsoletes-Dist because it would be respected only if all conflicting packages declare it, and resolve thus the "non-curation" of PyPi?

[edit] I don't know if pypa/packaging-problems#154 is more relevant to my question.