uv add rejects pytorch from its index when on mac

[baggiponte]

I am trying to use uv to add torch to my project's dependencies, but it fails. I am on macOS14.4.1 (ARM).

Here is a minimal repro.

[project]
name = "test"
version = "0.1.0"
requires-python = ">=3.10"

Then run:

uv python pin 3.10
uv add --extra-index-url="https://download.pytorch.org/whl/cpu" -- torch

Error:

warning: `uv add` is experimental and may change without warning.
Using Python 3.10.14 interpreter at: /Users/luca/Library/Application Support/uv/python/cpython-3.10.14-macos-aarch64-none/install/bin/python3
Creating virtualenv at: .venv
Resolved 13 packages in 1.69s
error: distribution torch==2.3.1+cpu @ registry+https://download.pytorch.org/whl/cpu can't be installed because it doesn't have a source distribution or wheel for the current platform

If I use uv pip install then everything works smoothly:

uv pip install --extra-index-url="https://download.pytorch.org/whl/cpu" -- torch
Resolved 9 packages in 1.73s
Installed 9 packages in 369ms
 + filelock==3.13.1
 + fsspec==2024.2.0
 + jinja2==3.1.3
 + markupsafe==2.1.5
 + mpmath==1.3.0
 + networkx==3.2.1
 + sympy==1.12
 + torch==2.3.1
 + typing-extensions==4.9.0

[charliermarsh]

This is already resolved on main but not released.

[charliermarsh]

Sorry, misread the last part of the error.

[charliermarsh]

Can you include the uv.lock that is generated?

[charliermarsh]

This one is pretty tricky... Basically, 2.3.1+cpu includes Windows and Linux wheels, but not macOS wheels. Separately, 2.3.1 (without the local version) has macOS wheels. We choose 2.3.1+cpu because we don't evaluate platform compatibility during a universal install. But then we don't end up including macOS wheels.

One path forward: if the user selects 2.3.1, always choose the non-local version. That at least gives users some more control.

I don't know what other options we have. Try to sniff platform compatibility on the wheel tags, at least for macOS / Windows / Linux?

[charliermarsh]

\cc @konstin

[charliermarsh]

I'd suggest using a direct URL for now while we figure out how to support this @baggiponte.

[charliermarsh]

I've confirmed that Poetry has the same problem. It also chooses:

[[package]]
name = "torch"
version = "2.3.1+cpu"
description = "Tensors and Dynamic neural networks in Python with strong GPU acceleration"
optional = false
python-versions = ">=3.8.0"
files = [
    {file = "torch-2.3.1+cpu-cp310-cp310-linux_x86_64.whl", hash = "sha256:d679e21d871982b9234444331a26350902cfd2d5ca44ce6f49896af8b3a3087d"},
    {file = "torch-2.3.1+cpu-cp310-cp310-win_amd64.whl", hash = "sha256:500bf790afc2fd374a15d06213242e517afccc50a46ea5955d321a9a68003335"},
    {file = "torch-2.3.1+cpu-cp311-cp311-linux_x86_64.whl", hash = "sha256:a272defe305dbd944aa28a91cc3db0f0149495b3ebec2e39723a7224fa05dc57"},
    {file = "torch-2.3.1+cpu-cp311-cp311-win_amd64.whl", hash = "sha256:d2965eb54d3c8818e2280a54bd53e8246a6bb34e4b10bd19c59f35b611dd9f05"},
    {file = "torch-2.3.1+cpu-cp312-cp312-linux_x86_64.whl", hash = "sha256:2141a6cb7021adf2f92a0fd372cfeac524ba460bd39ce3a641d30a561e41f69a"},
    {file = "torch-2.3.1+cpu-cp312-cp312-win_amd64.whl", hash = "sha256:6acdca2530462611095c44fd95af75ecd5b9646eac813452fe0adf31a9bc310a"},
    {file = "torch-2.3.1+cpu-cp38-cp38-linux_x86_64.whl", hash = "sha256:cab92d5101e6db686c5525e04d87cedbcf3a556073d71d07fbe7d1ce09630ffb"},
    {file = "torch-2.3.1+cpu-cp38-cp38-win_amd64.whl", hash = "sha256:dbc784569a367fd425158cf4ae82057dd3011185ba5fc68440432ba0562cb5b2"},
    {file = "torch-2.3.1+cpu-cp39-cp39-linux_x86_64.whl", hash = "sha256:a3cb8e61ba311cee1bb7463cbdcf3ebdfd071e2091e74c5785e3687eb02819f9"},
    {file = "torch-2.3.1+cpu-cp39-cp39-win_amd64.whl", hash = "sha256:df68668056e62c0332e03f43d9da5d4278b39df1ba58d30ec20d34242070955d"},
]

And then poetry install fails after poetry lock succeeds.

[baggiponte]

Can you include the uv.lock that is generated?

Ops, sorry for the delay. Here it is (sorry for the .txt format, github does not support it).

uv.lock.txt

I'd suggest using a direct URL for now while we figure out how to support this @baggiponte.

i.e. from pypi, right?

Thank you very much for the prompt support!

[dimbleby]

FWIW over in poetry I have mostly come to think that this is a torch problem and not a poetry problem at all. (And therefore also not a uv problem).

Specifically I reckon that pytorch/pytorch#110004 is the right request to make, and the right place to make it. "Closing as implemented" seems not correct, perhaps someone would like to try again.

More generally... pytorch does lots of unusual things with its multiple indexes and playing games with local version identifiers and suchlike. If you want a taste of the sort of fun uv is likely to have trying to accommodate all of this - and if you have plenty of time to spare - then see python-poetry/poetry#6409.

If uv - either deliberately or accidentally - were to make choices that somehow made sense of all of that, then there are plenty of folk in that thread who would likely be pleased to see it!

[baggiponte]

Uhm as an FYI: today I tried uv add torch on M3 mac and it correctly installed the version w/o CUDA (which was brought me to use uv add --extra-index-url="https://download.pytorch.org/whl/cpu" -- torch in the first place).

[dimbleby]

pytorch/pytorch#136275