Skip to content

Commit

Permalink
pypi-types: fix lenient requirement parsing (astral-sh#1529)
Browse files Browse the repository at this point in the history
This fixes a bug where `uv pip install` failed to install `polars`:

```
$ uv pip install polars==0.14.0
error: Failed to download: polars==0.14.0
  Caused by: Couldn't parse metadata of polars-0.14.0-cp37-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.whl from https://files.pythonhosted.org/packages/5c/c6/749022b096895790c971338de93e610210331ea6cb7c1c2cc32b7f433c4f/polars-0.14.0-cp37-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
  Caused by: Operator >= cannot be used with a wildcard version specifier
pyarrow>=4.0.*; extra == 'pyarrow'
       ^^^^^^^
```

Since `pyarrow>=4.0.*; extra == 'pyarrow'` is invalid *and* it comes
from the metadata of a dependency (that isn't under the control of the
end user), we actually attempt to "fix" it. Namely, wildcard
dependency specifications are only allowed with `==` and `!=`, as per
the [Version Specifiers spec]. (They aren't explicitly forbidden in
these cases, but instead only have specified behavior for the `==` and
`!=` operators.)

This is all fine, but it turns out that when we fix the `>=4.0.*`
component, we also strip the quotes around `pyarrow`. (Because some
dependency specifications include stray quotes.) We fix this by making
our quote stripping a bit more selective. (We require that it appear
adjacent to a digit or a `*`.)

Note that astral-sh#1477 also reports this error:

```
$ uv pip install 'requests>=2.30.*'
error: Failed to parse `requests>=2.30.*`
  Caused by: Operator >= cannot be used with a wildcard version specifier
requests>=2.30.*
```

However, we specifically keep that error message since it's something
under the end user's control. And similarly for a dependency
specification in a `requirements.txt` file.

Fixes astral-sh#1477

[Version Specifiers spec]:
https://packaging.python.org/en/latest/specifications/version-specifiers/
  • Loading branch information
BurntSushi authored Feb 16, 2024
1 parent 9737b93 commit a97c207
Show file tree
Hide file tree
Showing 2 changed files with 35 additions and 3 deletions.
6 changes: 3 additions & 3 deletions crates/pypi-types/src/lenient_requirement.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ static MISSING_COMMA: Lazy<Regex> = Lazy::new(|| Regex::new(r"(\d)([<>=~^!])").u
static NOT_EQUAL_TILDE: Lazy<Regex> = Lazy::new(|| Regex::new(r"!=~((?:\d\.)*\d)").unwrap());
/// Ex) `>=1.9.*`, `<3.4.*`
static INVALID_TRAILING_DOT_STAR: Lazy<Regex> =
Lazy::new(|| Regex::new(r"(<=|>=|<|>)(\d+(\.\d+)?)\.\*").unwrap());
Lazy::new(|| Regex::new(r"(<=|>=|<|>)(\d+(\.\d+)*)\.\*").unwrap());
/// Ex) `!=3.0*`
static MISSING_DOT: Lazy<Regex> = Lazy::new(|| Regex::new(r"(\d\.\d)+\*").unwrap());
/// Ex) `>=3.6,`
static TRAILING_COMMA: Lazy<Regex> = Lazy::new(|| Regex::new(r",\s*$").unwrap());
/// Ex) `>= '2.7'`, `>=3.6'`
static STRAY_QUOTES: Lazy<Regex> = Lazy::new(|| Regex::new(r#"['"]"#).unwrap());
static STRAY_QUOTES: Lazy<Regex> = Lazy::new(|| Regex::new(r#"['"]([*\d])|([*\d])['"]"#).unwrap());

/// Regex to match the invalid specifier, replacement to fix it and message about was wrong and
/// fixed
Expand All @@ -45,7 +45,7 @@ static FIXUPS: &[(&Lazy<Regex>, &str, &str)] = &[
// Given `>=3.6,`, rewrite to `>=3.6`
(&TRAILING_COMMA, r"${1}", "removing trailing comma"),
// Given `>= '2.7'`, rewrite to `>= 2.7`
(&STRAY_QUOTES, r"", "removing stray quotes"),
(&STRAY_QUOTES, r"$1$2", "removing stray quotes"),
];

fn parse_with_fixups<Err, T: FromStr<Err = Err>>(input: &str, type_name: &str) -> Result<T, Err> {
Expand Down
32 changes: 32 additions & 0 deletions crates/uv/tests/pip_install.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1097,3 +1097,35 @@ fn install_constraints_inline() -> Result<()> {

Ok(())
}

/// Tests that we can install `polars==0.14.0`, which has this odd dependency
/// requirement in its wheel metadata: `pyarrow>=4.0.*; extra == 'pyarrow'`.
///
/// The `>=4.0.*` is invalid, but is something we "fix" because it is out
/// of the control of the end user. However, our fix for this case ends up
/// stripping the quotes around `pyarrow` and thus produces an irrevocably
/// invalid dependency requirement.
///
/// See: <https://github.com/astral-sh/uv/issues/1477>
#[test]
fn install_pinned_polars_invalid_metadata() {
let context = TestContext::new("3.12");

// Install Flask.
uv_snapshot!(command(&context)
.arg("polars==0.14.0"),
@r###"
success: true
exit_code: 0
----- stdout -----
----- stderr -----
Resolved 1 package in [TIME]
Downloaded 1 package in [TIME]
Installed 1 package in [TIME]
+ polars==0.14.0
"###
);

context.assert_command("import polars").success();
}

0 comments on commit a97c207

Please sign in to comment.