Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bump Python and RAPIDS versions #16

Merged
merged 11 commits into from
Apr 2, 2024
Merged

Bump Python and RAPIDS versions #16

Changes from 5 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
055ffe1
Bump Python version
ryantwolf Mar 27, 2024
6dd63c0
Change pinned rapids to lower bound
ryantwolf Mar 27, 2024
c74743a
Remove wildcard suffix
ryantwolf Mar 27, 2024
f850264
Only bump dask-cuda
ryantwolf Mar 27, 2024
71ae79e
Update min rapids version
ryantwolf Mar 27, 2024
e6d170f
Use latest RAPIDS versions
ryantwolf Mar 27, 2024
248f5c5
Fix notation
ryantwolf Mar 27, 2024
0aeca01
Move to bounds
ryantwolf Mar 27, 2024
2d283ae
Free space on test worker
ryantwolf Apr 1, 2024
b38b8fc
Merge branch 'main' into rywolf/version-bump
ryantwolf Apr 2, 2024
411f946
Add python version in readme
ryantwolf Apr 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
"Programming Language :: Python :: 3",
],
packages=find_packages(),
python_requires=">=3.7",
python_requires=">=3.10",
install_requires=[
"dask[complete]>=2021.7.1",
"distributed>=2021.7.1",
Expand All @@ -55,10 +55,10 @@
"comment_parser",
"beautifulsoup4",
"mwparserfromhell @ git+https://github.com/earwig/mwparserfromhell.git@0f89f44",
"cudf-cu12==23.10.*",
"dask-cudf-cu12==23.10.*",
"cugraph-cu12==23.10.*",
"dask-cuda==23.10.*",
"cudf-cu12==23.12.*",
"dask-cudf-cu12==23.12.*",
"cugraph-cu12==23.12.*",
"dask-cuda==23.12.*",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want to bump this way up to 24.02 which is the latest stable RAPIDS release. (See release blog link)

Suggested change
"cudf-cu12==23.12.*",
"dask-cudf-cu12==23.12.*",
"cugraph-cu12==23.12.*",
"dask-cuda==23.12.*",
"cudf-cu12==24.02.*",
"dask-cudf-cu12==24.02.*",
"cugraph-cu12==24.02.*",
"dask-cuda==24.02.*",

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a clean way to pull of a lower bound on the RAPIDS libraries so we don't have to manually update the versions? Not sure if that's good practice though. In the "Remove wildcard suffix" commit I tried to do it but got these errors:

cugraph 23.12.0 requires dask-cuda==23.12.*, but you have dask-cuda 24.2.0 which is incompatible.
cugraph 23.12.0 requires rapids-dask-dependency==23.12.*, but you have rapids-dask-dependency 24.2.0 which is incompatible.
cugraph-service-server 23.12.0 requires dask-cuda==23.12.*, but you have dask-cuda 24.2.0 which is incompatible.
cugraph-service-server 23.12.0 requires rapids-dask-dependency==23.12.*, but you have rapids-dask-dependency 24.2.0 which is incompatible.
cuml 23.12.0 requires dask-cuda==23.12.*, but you have dask-cuda 24.2.0 which is incompatible.
cuml 23.12.0 requires rapids-dask-dependency==23.12.*, but you have rapids-dask-dependency 24.2.0 which is incompatible.
dask-cudf 23.12.0 requires rapids-dask-dependency==23.12.*, but you have rapids-dask-dependency 24.2.0 which is incompatible.
raft-dask 23.12.0 requires dask-cuda==23.12.*, but you have dask-cuda 24.2.0 which is incompatible.
raft-dask 23.12.0 requires rapids-dask-dependency==23.12.*, but you have rapids-dask-dependency 24.2.0 which is incompatible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure why your lower bound commit failed, trying again locally to see if i can repro.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, below seems to be working with base environment python=3.10

Currently it seems like for RAPIDS pip installation is supported in Python 3.9 and 3.10.
The nightly builds also support 3.11, so in the next release that should work too.

image 
diff --git a/setup.py b/setup.py
index ae51575..0f4fc95 100644
--- a/setup.py
+++ b/setup.py
@@ -36,7 +36,7 @@ setup(
         "Programming Language :: Python :: 3",
     ],
     packages=find_packages(),
-    python_requires=">=3.7",
+    python_requires=">=3.10",
     install_requires=[
         "dask[complete]>=2021.7.1",
         "distributed>=2021.7.1",
@@ -55,15 +55,16 @@ setup(
         "comment_parser",
         "beautifulsoup4",
         "mwparserfromhell @ git+https://github.com/earwig/mwparserfromhell.git@0f89f44",
-        "cudf-cu12==23.10.*",
-        "dask-cudf-cu12==23.10.*",
-        "cugraph-cu12==23.10.*",
-        "dask-cuda==23.10.*",
+        "cudf-cu12>=24.2",
+        "dask-cudf-cu12>=24.2",
+        "cugraph-cu12>=24.2",
+        "dask-cuda>=24.2",
         "spacy>=3.6.0, <4.0.0",
         "presidio-analyzer==2.2.351",
         "presidio-anonymizer==2.2.351",
         "usaddress==0.5.10",
         "nemo_toolkit[nlp]>=1.23.0",
+       
     ],
     entry_points={
         "console_scripts": [

Copy link
Collaborator

@VibhuJawa VibhuJawa Mar 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should set python_requires==3.10 and then once rapids releases (which should be on Wed, Apr 12, 2023)we can bump it to >=.

The Python 3.11 support landed in this release (rapidsai/build-planning#3).

"spacy>=3.6.0, <4.0.0",
"presidio-analyzer==2.2.351",
"presidio-anonymizer==2.2.351",
Expand Down