Poetry fails resolving PyTorch on MacOS

[ralbertazzi]

• Poetry version: 1.5.0 dev (post poetry-core 1.6 PR)
• Python version: 3.9
• OS version and name: MacOS Monterey 12.5.1, Apple M1
• pyproject.toml:

[tool.poetry]
authors = ["Riccardo Albertazzi <email@email.com>"]
description = "Hello world"
name = "torch-project"
version = "0.1.0"

[[tool.poetry.source]]
name = "torch-cpu"
priority = "explicit"
url = "https://download.pytorch.org/whl/cpu"

[tool.poetry.dependencies]
python = "~3.9"

torch = { version = "2.0.1", source = "torch-cpu" }

[build-system]
build-backend = "poetry.core.masonry.api"
requires = ["poetry-core>=1.5.0"]

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate. Possibly related: PEP 440 version matching of local version identifier clause (e.g. 1.2.3+local) not implemented correctly in solver #2543
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Configuration file exists at /Users/ra/Library/Preferences/pypoetry, reusing this directory.

Consider moving TOML configuration files to /Users/ra/Library/Application Support/pypoetry, as support for the legacy directory will be removed in an upcoming release.
Loading configuration file /Users/ra/Library/Preferences/pypoetry/config.toml
Loading configuration file /Users/ra/Library/Preferences/pypoetry/auth.toml
Adding repository torch-cpu (https://download.pytorch.org/whl/cpu) and setting it as explicit
Using virtualenv: /Users/ra/projects/poetry/.venv
Updating dependencies
Resolving dependencies...
   1: fact: torch-project is 0.1.0
   1: derived: torch-project
   1: fact: torch-project depends on torch (2.0.1)
   1: selecting torch-project (0.1.0)
   1: derived: torch (==2.0.1)
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
Creating new session for download.pytorch.org
[urllib3.connectionpool] Starting new HTTPS connection (1): download.pytorch.org:443
[urllib3.connectionpool] https://download.pytorch.org:443 "GET /whl/cpu/torch/ HTTP/1.1" 200 64566
Source (torch-cpu): 2 packages found for torch 2.0.1
   1: selecting torch (2.0.1+cpu)
   1: Version solving took 0.784 seconds.
   1: Tried 1 solutions.

Finding the necessary packages for the current system
Source (poetry-repo): 1 packages found for torch 2.0.1

Package operations: 1 install, 0 updates, 0 removals

  • Installing torch (2.0.1+cpu): Pending...
Skipping wheel torch-2.0.1+cpu-cp310-cp310-linux_x86_64.whl as this is not supported by the current environment
Skipping wheel torch-2.0.1+cpu-cp310-cp310-win_amd64.whl as this is not supported by the current environment
Skipping wheel torch-2.0.1+cpu-cp311-cp311-linux_x86_64.whl as this is not supported by the current environment
Skipping wheel torch-2.0.1+cpu-cp311-cp311-win_amd64.whl as this is not supported by the current environment
Skipping wheel torch-2.0.1+cpu-cp38-cp38-linux_x86_64.whl as this is not supported by the current environment
Skipping wheel torch-2.0.1+cpu-cp38-cp38-win_amd64.whl as this is not supported by the current environment
Skipping wheel torch-2.0.1+cpu-cp39-cp39-linux_x86_64.whl as this is not supported by the current environment
Skipping wheel torch-2.0.1+cpu-cp39-cp39-win_amd64.whl as this is not supported by the current environment
  • Installing torch (2.0.1+cpu): Failed

  Stack trace:

  5  ~/projects/poetry/src/poetry/installation/executor.py:286 in _execute_operation
      284│ 
      285│             try:
    → 286│                 result = self._do_execute_operation(operation)
      287│             except EnvCommandError as e:
      288│                 if e.e.returncode == -2:

  4  ~/projects/poetry/src/poetry/installation/executor.py:388 in _do_execute_operation
      386│             return 0
      387│ 
    → 388│         result: int = getattr(self, f"_execute_{method}")(operation)
      389│ 
      390│         if result != 0:

  3  ~/projects/poetry/src/poetry/installation/executor.py:508 in _execute_install
      506│ 
      507│     def _execute_install(self, operation: Install | Update) -> int:
    → 508│         status_code = self._install(operation)
      509│ 
      510│         self._save_url_reference(operation)

  2  ~/projects/poetry/src/poetry/installation/executor.py:546 in _install
      544│             archive = self._download_link(operation, Link(package.source_url))
      545│         else:
    → 546│             archive = self._download(operation)
      547│ 
      548│         operation_message = self.get_operation_message(operation)

  1  ~/projects/poetry/src/poetry/installation/executor.py:736 in _download
      734│ 
      735│     def _download(self, operation: Install | Update) -> Path:
    → 736│         link = self._chooser.choose_for(operation.package)
      737│ 
      738│         if link.yanked:

  RuntimeError

  Unable to find installation candidates for torch (2.0.1+cpu)

  at ~/projects/poetry/src/poetry/installation/chooser.py:76 in choose_for
       72│ 
       73│             links.append(link)
       74│ 
       75│         if not links:
    →  76│             raise RuntimeError(f"Unable to find installation candidates for {package}")
       77│ 
       78│         # Get the best link
       79│         chosen = max(links, key=lambda link: self._sort_key(package, link))
       80│

Issue

Poetry fails resolving Poetry on MacOS and I think this is due to local identifiers. I don't know what's the right course of action here so I'm asking support from the core maintainers.

The PyTorch repository at https://download.pytorch.org/whl/cpu/torch contains:

• Wheels for PyTorch 2.0.1 for MacOS without local identifier. E.g. torch-2.0.1-cp39-none-macosx_11_0_arm64.whl, which is the one I'd like to resolve and install
• Wheels for PyTorch 2.0.1 for Linux with local identifier. E.g. torch-2.0.1+cpu-cp39-cp39-linux_x86_64.whl

The issue seems in how the presence/absence of local identifiers results in separate versions being treated by Poetry. The resolver in fact selects 2.0.1+cpu as the best fitting version and the executor can't (rightfully) find a package that for MacOS that matches the selected version.

[radoering]

The issue is that "2.0.1" is equal to "==2.0.1" which allows local versions and since local versions are considered greater than non local version they will be preferred. That's all conform to PEP 440. In general, you will never get a non local version of a package from a source that also provides a local version. The only escape hatch is arbitrary equality, which is not supported by poetry. See #7256.

Even though the use of arbitrary equality is discouraged, I think in this case it's the most sensible option. I could imagine to introduce an ExactVersion in poetry.core.constraints.version with the appropriate behavior.

For now, the obvious workaround is to use an URL dependency.

[dimbleby]

#6150, #7597

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.