Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] mvn resolve dependencies failed missing rapids-4-spark-sql-plugin-api_2.12 of 311 shim #9214

Closed
pxLi opened this issue Sep 11, 2023 · 2 comments
Assignees
Labels
bug Something isn't working build Related to CI / CD or cleanly building

Comments

@pxLi
Copy link
Collaborator

pxLi commented Sep 11, 2023

Describe the bug
related to #9148

mvn get or some other dependency commands would fail missing com.nvidia:rapids-4-spark-sql-plugin-api_2.12:jar:spark311:23.10.0-SNAPSHOT while try resolve plugin artifact dependencies.

00:41:51  [INFO] ------------------------------------------------------------------------
00:41:51  [ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:2.8:get (default-cli) on project rapids-4-spark-parent: Couldn't download artifact: Missing:
00:41:51  [ERROR] ----------
00:41:51  [ERROR] 1) com.nvidia:rapids-4-spark-sql-plugin-api_2.12:jar:spark311:23.10.0-SNAPSHOT
00:41:51  [ERROR] 
00:41:51  [ERROR] Try downloading the file manually from the project website.
00:41:51  [ERROR] 
00:41:51  [ERROR] Then, install it using the command:
00:41:51  [ERROR] mvn install:install-file -DgroupId=com.nvidia -DartifactId=rapids-4-spark-sql-plugin-api_2.12 -Dversion=23.10.0-SNAPSHOT -Dclassifier=spark311 -Dpackaging=jar -Dfile=/path/to/file
00:41:51  [ERROR] 
00:41:51  [ERROR] Alternatively, if you host your own repository you can deploy the file there:
00:41:51  [ERROR] mvn deploy:deploy-file -DgroupId=com.nvidia -DartifactId=rapids-4-spark-sql-plugin-api_2.12 -Dversion=23.10.0-SNAPSHOT -Dclassifier=spark311 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

Steps/Code to reproduce bug
e.g.

mvn org.apache.maven.plugins:maven-dependency-plugin:2.8:get -B -Dmaven.repo.local=<path>/.m2 -DrepoUrl=https://urm.nvidia.com/artifactory/sw-spark-maven -Ddest=<path>/jars -DgroupId=com.nvidia -DartifactId=rapids-4-spark_2.12 -Dversion=23.10.0-SNAPSHOT

or

mvn -U -B -s jenkins/settings.xml -P mirror-apache-to-urm -Dmaven.repo.local=<path>/.m2/repository dependency:go-offline dependency:resolve-plugins -Dcuda.version=cuda11

Expected behavior
it should pass the mvn get

@pxLi pxLi added bug Something isn't working ? - Needs Triage Need team to review and classify labels Sep 11, 2023
@pxLi
Copy link
Collaborator Author

pxLi commented Sep 11, 2023

all other shims API artifacts are available on maven repo, but not 311 (I am not sure if its intentional or not)

also cc @gerashegalov to help take a look, thanks

@gerashegalov gerashegalov removed the ? - Needs Triage Need team to review and classify label Sep 12, 2023
@gerashegalov
Copy link
Collaborator

The nightly pipeline resumed deploying all necessary artifacts after fixing #9213

I verified manually that both commands work on an empty local m2 repo.

Moreover mvn org.apache.maven.plugins:maven-dependency-plugin:2.8:get -DartifactId=rapids-4-spark_2.12 would have worked even prior to fixing #9213 had it been executed outside the project directory, because it has no dependency on anything com.nvidia other than a parent pom referencei

@gerashegalov gerashegalov added the build Related to CI / CD or cleanly building label Sep 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working build Related to CI / CD or cleanly building
Projects
None yet
Development

No branches or pull requests

2 participants