Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rip does not follow spec for handling of pre-releases #118

Closed
notatallshaw opened this issue Dec 6, 2023 · 18 comments · Fixed by #141
Closed

rip does not follow spec for handling of pre-releases #118

notatallshaw opened this issue Dec 6, 2023 · 18 comments · Fixed by #141
Assignees
Labels
bug Something isn't working enhancement New feature or request

Comments

@notatallshaw
Copy link

notatallshaw commented Dec 6, 2023

Currently rip does not accept a pre-release if there are no other packages, but the documentation states the follow: https://packaging.python.org/en/latest/specifications/version-specifiers/#handling-of-pre-releases

By default, dependency resolution tools SHOULD: ... accept remotely available pre-releases for version specifiers where there is no final or post release that satisfies the version specifier

But rip excludes all pre-releases even if nothing else is available:

e.g. cargo r -- opentelemetry-exporter-prometheus rusults in:

  × Could not solve for the requested requirements:
  │ The following packages are incompatible
  │ |-- opentelemetry-exporter-prometheus * cannot be installed because there are no viable options:
  │ 	|-- opentelemetry-exporter-prometheus 1.12.0rc1 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 1.10a0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.42b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.41b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.40b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.39b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.38b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.37b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.36b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.35b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.34b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.33b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.32b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.30b1 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.30b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.29b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.17b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.16b1 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.16b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.15b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.14b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.13b0 is excluded because prereleases are not allowed
  │ 	|-- opentelemetry-exporter-prometheus 0.12b0 is excluded because prereleases are not allowed

In my opinion there should be an option to force exclude pre release packages and keep this behavior, but this does come up in the Python ecosystem naturally: #74 (comment)

By the way, my understanding is Pip's behavior is to do this when collecting packages, but not when backtracking transitive dependencies, I have a question on the Python board if this is really following the spec: https://discuss.python.org/t/handeling-of-pre-releases-when-backtracking/40505

@notatallshaw notatallshaw changed the title rip does not follow spec for handeling packages with only pre-releases rip does not follow spec for handling of pre-releases Dec 7, 2023
@notatallshaw
Copy link
Author

notatallshaw commented Dec 7, 2023

FYI, the discussion on the thread questions the behavior I have described for Pip: https://discuss.python.org/t/handling-of-pre-releases-when-backtracking/40505/4

I will try and reproduce, but I will do this seperarelty from rip and file a Pip bug if I find the behavior does not match the comment I linked in all cases.

@tdejager
Copy link
Contributor

Yes we should fix this definitely we just have not come around to it yet!

@notatallshaw
Copy link
Author

notatallshaw commented Jan 10, 2024

Some updates on the Python discussion board: https://discuss.python.org/t/handling-of-pre-releases-when-backtracking/40505/10

It's clear to me now that what Pip does here "selects a pre-release if all releases are pre-releases" and what the spec says are different.

@wolfv if any RIP maintainers would like to provide any feedback there on any changes they would like to see here (Pip documentation on what they do, update spec to Pip or something different, change in Pip behavior to the spec, etc.) I'm sure input would be appreciated.

@wolfv
Copy link
Member

wolfv commented Jan 10, 2024

Thanks @notatallshaw for pointing this out. To be honest, what pip implements would be quite straightforward to implement for us, too. So we might start out with that :)

The --pre flag to get all possible pre-releases would also continue to work.

@notatallshaw
Copy link
Author

Sounds good to me, and sounds like the spec should be updated. Although, when I get a chance, I will check what Poetry does.

I would still strongly prefer there be an option to exclude all pre-releases. I would really like to not be surprised in prod that a pre-release slipped in. But maybe this use case is too niche.

@wolfv
Copy link
Member

wolfv commented Jan 10, 2024

We have that as an option in the enum:

/// Defines how to pre-releases are handled during package resolution.
#[derive(Default, Debug, Clone, Copy, Eq, PartialOrd, PartialEq)]
pub enum PreReleaseResolution {
    /// Don't allow pre-releases to be selected during resolution
    Disallow,

    /// Allow pre-releases to be selected during resolution but only if there are no other versions
    /// available (default)
    #[default]
    AllowIfNoOtherVersions,

    /// Allow pre-releases to be selected during resolution
    Allow,
}

It might be the better default ... although, as you noted, will prevent some packages from resolving.

@baszalmstra
Copy link
Contributor

Hey @notatallshaw ! Thanks for pitching and for the link! I would also be very interested in what Poetry does!

@notatallshaw
Copy link
Author

As best as I can tell, Poetry appears to follow the spec for direct dependencies, but struggles when resolving transative dependencies: https://discuss.python.org/t/handling-of-pre-releases-when-backtracking/40505/11

@tdejager
Copy link
Contributor

tdejager commented Jan 11, 2024

So if we want to follow pip's rules, which rules should we follow? I'm getting a bit lost in the details here. I initially wrote down: #4 but not sure where I got that idea from anymore.

Then there's @pradyun comment:

  • There are only pre-release versions for a project.
  • The version specifier is a pre-release specifier.
  • --pre was passed.

Which seem quite reasonable.

But @notatallshaw you've also noted in: https://discuss.python.org/t/handling-of-pre-releases-when-backtracking/40505/11 that pip either does or does not use pre-releases when backtracking? Which kind-of comes back to #4. But also does not make a whole lot of sense to me. Although poetry does not consider transitive pre-release dependencies?

Maybe @wolfv @notatallshaw could clearly lay out the rules that pip follows in this thread or in the linked one so that we have a written down reference to follow, instead of just code :)

@wolfv
Copy link
Member

wolfv commented Jan 11, 2024

I would like to follow what Pradyun laid out.

  • If a package only has pre-releases, use them.
  • If a version specifier contains some notion of pre-release (e.g. jupyterlab==4.1.0b0 then use the pre-release. Or toml>=4.0.1b0. This is easy for us to implement because this means just a "local" check in the comparison operator.

Although there will be some trickery involved in implementing the first rule while also implementing the second rule, so let's see (since we cannot just filter out the pre-releases anymore).

@tdejager
Copy link
Contributor

So what we discussed what we could do is to change the contains of the PyPiVersionSet to have global information regarding if pre-releases are allowed for that VersionSet either by the consequence of there only being pre-releases for that package version, allowing all pre-releases by the user, or by being explicitely selected e.g. >1.0a0.

Note that the the pep440_rs crate includes the matching of pre-releases by default so:

let pre = pep440_rs::Version::from_str("1.0.0a1").unwrap();
let set = pep440_rs::VersionSpecifier::from_str(">=0.9").unwrap();
assert!(set.contains(&pre)); // assertion will pass

So take care when implementing this :)

E.g if you have 2 packages

  • A depends on foo==1.0a0
  • B depends on foo>=0.9

If we first process A and the pre-releases are added for foo, and we subsequently process B if we naively match, it would be totally fine using 1.0.a0 because according pep440_rs the versionset contains this version. This might not be what we want, at least that's the gist I get from your last comment @wolfv .

If you have a lot of problems with this you can always implement contains for the pep440_rs::Version yourself, and explicitely exclude pre-releases.

@tdejager
Copy link
Contributor

tdejager commented Jan 11, 2024

@notatallshaw The final point of unclarity for me remains in the comment in the python discourse thread w.r.t backtracking and transitive dependencies, including the poetry comment. Hoping you can clarify :)!

So in my head there are two cases:

  1. Transitive dependencies that run into pre-releases a 1.0 -> b 1.0 -> c 1.0a
  2. During backtracking, deciding that you need a new version and at that point running into pre-releases.

But I suppose in both cases you should just look at which candidates would match the VersionSet, so I don't really see why one would make a difference to how that works when compared to direct dependencies.

That is if the direct dependency foo >= 1.0.0 does not select pre-releases (disregarding the case for this example when there are only pre-releases) then a -> foo where foo >= 1.0.0 Should also not select pre-releases. Alternatively, if it does select them for direct dependencies it should also do so for both case 1 and 2.

However, if I read your posts correctly it seems that:

  • Poetry does not select pre-releeases for case (1).
  • Pip does not select pre-releases for case (2). Although this needs verification?

Am I correct in my reading of this?

@notatallshaw
Copy link
Author

notatallshaw commented Jan 11, 2024

I'm a bit confused by the different questions, let me outline the scenario I was thinking about when I originally started that thread:

  • Requirements: a == 1.0, b == 1.0 - For simplicity these are the only versions available
  • a -> c != 1.0
  • b -> c != 2.0
  • Available versions of c: 1.0, 2.0, and 3.0a1

I thought this case was special for Pip when I opened the thread because I thought Pip followed the spec. But now I think (but haven't tested) that this case is not special for Pip because for Pip this is no different than normal, Pip see's C as having non-prerelease versions and none of the requirements involve a pre-release specifier so Pip will not choose 3.0a1. So this is exactly in-line with Pip's expectations.

I think Poetry also doesn't choose 3.0a1 for C but considers it a bug, but I haven't tested, I'm just going off from what I read on their issue tracker. Possibly it's far more nuanced.

I still don't think Pip is consistent in how it handles this, the real world motivation for me initially thinking about this was a user reported an issue to Pip that they were gets a ResolutionImpossible error. They were on Apple Silicon and if they installed tensorflow directly it worked with a pre-release version (the only version available in that environment), but when tensorflow was a transitive dependency they got the ResolutionImpossible error. If someone is interested and has an Apple Silicon machine, they can use PyPI timemachine to reproduce: pypa/pip#12049

Finally, and I think your questions are getting at this (?), I have not even thought about the scenario where one or both of the specifiers include a pre-release:

  • Requirements: a == 1.0, b == 1.0 - For simplicity this is the only versions available
  • a -> c > 2.0a1
  • b -> c != 2.0
  • Available versions of c: 1.0, 2.0, and 3.0a1

I would suspect Pip's behavior is the same as to whether it is a transitive dependency or not and depends on the order it is checking the specifiers.

pandas>2.1.4 pandas>=2.1.5rc0

pip install --dry-run --no-deps --ignore-install "pandas>2.1.4" "pandas>=2.1.5rc0"
ERROR: Could not find a version that satisfies the requirement pandas>2.1.4 (from versions: 0.1, 0.2, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.21.1, 0.22.0, 0.23.0, 0.23.1, 0.23.2, 0.23.3, 0.23.4, 0.24.0, 0.24.1, 0.24.2, 0.25.0, 0.25.1, 0.25.2, 0.25.3, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.4.0rc0, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.5.0rc0, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 2.0.0rc0, 2.0.0rc1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0rc0, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0rc0)
ERROR: No matching distribution found for pandas>2.1.4

pandas>=2.1.5rc01 pandas>2.1.4

pip install --dry-run --no-deps --ignore-install "pandas>=2.1.5rc01" "pandas>2.1.4" 
Collecting pandas>=2.1.5rc01
  Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.0 MB)
Would install pandas-2.2.0rc0

And which specifier operators are being used:

pandas>2.1.5rc01

pip install --dry-run --no-deps --ignore-install "pandas>2.1.5rc01"
ERROR: Could not find a version that satisfies the requirement pandas>2.1.5a1 (from versions: 0.1, 0.2, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.21.1, 0.22.0, 0.23.0, 0.23.1, 0.23.2, 0.23.3, 0.23.4, 0.24.0, 0.24.1, 0.24.2, 0.25.0, 0.25.1, 0.25.2, 0.25.3, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.4.0rc0, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.5.0rc0, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 2.0.0rc0, 2.0.0rc1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0rc0, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0rc0)
ERROR: No matching distribution found for pandas>2.1.5rc01

pandas>=2.1.5rc01

pip install --dry-run --no-deps --ignore-install "pandas>=2.1.5rc01"
Collecting pandas>=2.1.5rc01
  Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.0 MB)
Would install pandas-2.2.0rc0

Off to report more issues to the Pip tracker.

@tdejager
Copy link
Contributor

tdejager commented Jan 11, 2024

Finally, and I think your questions are getting at this (?), I have not even thought about the scenario where one or both of the specifiers include a pre-release:

* Requirements: `a == 1.0, b == 1.0` - For simplicity this is the only versions available

* `a -> c > 2.0a1`

* `b -> c != 2.0`

* Available versions of c: `1.0`, `2.0`, and `3.0a1`

I would suspect Pip's behavior is the same as to whether it is a transitive dependency or not and depends on the order it is checking the specifiers.

pandas>2.1.4 pandas>=2.1.5rc0

pip install --dry-run --no-deps --ignore-install "pandas>2.1.4" "pandas>=2.1.5rc0"
ERROR: Could not find a version that satisfies the requirement pandas>2.1.4 (from versions: 0.1, 0.2, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.21.1, 0.22.0, 0.23.0, 0.23.1, 0.23.2, 0.23.3, 0.23.4, 0.24.0, 0.24.1, 0.24.2, 0.25.0, 0.25.1, 0.25.2, 0.25.3, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.4.0rc0, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.5.0rc0, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 2.0.0rc0, 2.0.0rc1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0rc0, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0rc0)
ERROR: No matching distribution found for pandas>2.1.4

pandas>=2.1.5rc01 pandas>2.1.4

pip install --dry-run --no-deps --ignore-install "pandas>=2.1.5rc01" "pandas>2.1.4" 
Collecting pandas>=2.1.5rc01
  Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.0 MB)
Would install pandas-2.2.0rc0

And which specifier operators are being used:

pandas>2.1.5rc01

pip install --dry-run --no-deps --ignore-install "pandas>2.1.5rc01"
ERROR: Could not find a version that satisfies the requirement pandas>2.1.5a1 (from versions: 0.1, 0.2, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.21.1, 0.22.0, 0.23.0, 0.23.1, 0.23.2, 0.23.3, 0.23.4, 0.24.0, 0.24.1, 0.24.2, 0.25.0, 0.25.1, 0.25.2, 0.25.3, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.4.0rc0, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.5.0rc0, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 2.0.0rc0, 2.0.0rc1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0rc0, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0rc0)
ERROR: No matching distribution found for pandas>2.1.5rc01

pandas>=2.1.5rc01

pip install --dry-run --no-deps --ignore-install "pandas>=2.1.5rc01"
Collecting pandas>=2.1.5rc01
  Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.0 MB)
Would install pandas-2.2.0rc0

Off to report more issues to the Pip tracker.

Wow, I would expect that in both the cases pandas>2.1.4 pandas>=2.1.5rc0 and pandas>=2.1.5rc01 pandas>2.1.4 it should not have a solution. Is that also what you would expect?

Also, the second example with the specifier caught me off-guard as well.

@notatallshaw
Copy link
Author

Wow, I would expect that in both the cases pandas>2.1.4 pandas>=2.1.5rc0 and pandas>=2.1.5rc01 pandas>2.1.4 it should not have a solution. Is that also what you would expect?

I thought both of them should have a solution, but either way I would prefer consistency.

Also, the second example with the specifier caught me off-guard as well.

Some further discussion on Pip's state of affairs on one of the issues I raised: pypa/pip#12471 (comment)

IMO Pip's situation is the following:

  • It isn't following the spec
  • It isn't trying to follow the spec
  • It's intended behavior isn't documented
  • It's intended behavior isn't tested for
  • It isn't following it's intended behavior

But, as per the discussion I link to, this is all likely because the logic was implemented before there was a spec, and there hasn't been a significant interest in working on this aspect of requirement resolution.

Fow now I think RIP is going to need to make it's own choices, document it's own intended behavior, and take a pragmatical approach of what works in the Python ecosystem.

@pfmoore
Copy link

pfmoore commented Jan 11, 2024

IMO Pip's situation is the following:

  • It isn't following the spec
  • It isn't trying to follow the spec
  • It's intended behavior isn't documented
  • It's intended behavior isn't tested for
  • It isn't following it's intended behavior

Pip maintainer here - I disagree with this explanation.

IMO, pip isn't following the spec, but it should be. That's a bug in pip. That's all. It's not a bug we'll be able to fix quickly (so if rip follows the spec, it will behave differently than pip in some edge cases, probably for some time) but it is "just" a bug.

Talk of pip's "intended behaviour" basically comes down to a couple of pip maintainers (one of whom was me) having mentioned what they thought pip did as part of a discussion. It was a statement of what we think the code does, not a claim that it's intentional (much less that it's intentionally different to the spec!)

@notatallshaw
Copy link
Author

notatallshaw commented Jan 11, 2024

Talk of pip's "intended behaviour" basically comes down to a couple of pip maintainers (one of whom was me) having mentioned what they thought pip did as part of a discussion. It was a statement of what we think the code does, not a claim that it's intentional (much less that it's intentionally different to the spec!)

Apologies, it seems I misinterpreted some statements.

IMO, pip isn't following the spec, but it should be. That's a bug in pip. That's all. It's not a bug we'll be able to fix quickly (so if rip follows the spec, it will behave differently than pip in some edge cases, probably for some time) but it is "just" a bug.

I have been confused on this point over the course of this conversation, because of you saying that the spec says SHOULD and MAY that Pip could be interpreted as not violating the spec (which, in some sense, I am taking to have an equivelence as "following the spec").

I think this, with an outlining of behavior of what the code in Pip does, and the highlighting that Pip may have implemented this behavior before there was a spec, is what led to my mistaken assumption of Pip's "intended" behavior.

@tdejager
Copy link
Contributor

tdejager commented Jan 12, 2024

Nice to see a fruitful discussion coming out of this. Also both @notatallshaw and @pfmoore, thank you for all the hard work you are doing, I've really learned a lot about the pypa/pypi packaging space these last month, and all of your help has been invaluable. 🙏 😃

I think for now we will be going with an implementation that is close to pip, and disregards some of the spec, of course we would rather follow it, but I have the feeling it might change in the coming time. @wolfv is working on an implementation already: #141

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
5 participants