Remove cuda-python and cudatoolkit from rapids metapackage. #695
Conversation
| @@ -29,12 +29,6 @@ requirements: | |||
| - cuda-version ={{ cuda_version }} | |||
| run: | |||
| - {{ pin_compatible('cuda-version', max_pin='x', min_pin='x') }} | |||
| {% if cuda_major == "11" %} | |||
| - cuda-python {{ cuda11_cuda_python_version }} | |||
| - cudatoolkit | |||
There was a problem hiding this comment.
Just checking... was removing cudatoolkit for CUDA 11.x packages also intentional here?
There was a problem hiding this comment.
I saw this in the description:
I also removed cudatoolkit because that is a transitive dependency of RAPIDS libraries, and only needed for CUDA 11
But this is in a block that's CUDA-11-specific.
There was a problem hiding this comment.
Yes, it was intentional. We can revert that change if we want to keep this PR narrower, but it falls into the same bucket of "don't put packages in the rapids metapackage if we expect them to be supplied transitively by RAPIDS libraries, because it has the potential to make the solver do bad things with its priorities."
There are other packages that also fall in this category, like ptxcompiler (which is a hard dependency of cudf's CUDA 11 package).
The rapids metapackage (imo) should be only RAPIDS libraries, plus any other constraints needed to make the environment simpler for conda/mamba to resolve or more reproducible. If others agree, we could remove things like numba, numpy, etc. and rely on RAPIDS libraries to pin their own compatible versions of those. We certainly don't want the rapids metapackage to make the pinnings for packages like those any tighter than what RAPIDS libraries collectively support.
Changes of that scale would require careful testing, and wouldn't be suitable for 24.02.
There was a problem hiding this comment.
Ok thank you for that explanation! Just wanted to double-check that this was intentional, and that I understood it fully.
|
/merge |
Fixes attempted in rapidsai/integration#695 & rapidsai/integration#697 don't play well with CEC at large, but this repo can pin `cuda-python` more restrictively. This PR pins `cuda-python` to the CUDA `major.minor.*` version. See also conda-forge/cuda-python-feedstock#66 Authors: - Ray Douglass (https://github.com/raydouglass) Approvers: - Bradley Dice (https://github.com/bdice) - Jake Awe (https://github.com/AyodeAwe) URL: #624
We observed that some CUDA packages were being pulled from the
nvidiachannel here: https://github.com/rapidsai/docker/actions/runs/7709797316/job/21011757079#step:9:552I found that removing
cuda-pythonchanges the solver behavior in a way that satisfies our needs.I also removed
cudatoolkitbecause that is a transitive dependency of RAPIDS libraries, and only needed for CUDA 11. It is not needed at the top levelrapidsmetapackage. Likecuda-python, it is constrained bycuda-version, so I think it's better to cut it.Details:
The explicit dependency
rapids -> cuda-pythonis forcing the solver to prefer newercuda-pythonversions, which in turn makescuda-version=12.0ineffective at constrainingcuda-cudart. The solver prefers to takecuda-python 12.3from conda-forge, but it also has to havecuda-version 12.0per our constraints (cuda-python 12.3requirescuda-cudart>=12.3.101,<13.0a0andcuda-version>=12.0,<13.0a0). So then we getnvidiachannel packages (which are not constrained bycuda-version) ofcuda-cudart 12.3to satisfycuda-python 12.3rather than installcuda-python 12.0and getcuda-cudartfrom conda-forge.