Add DatabricksShimVersion to unshimmed class list [databricks] #4921
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Raza Jafri <rjafri@nvidia.com>
|
@pxLi PTAL |
|
build |
Signed-off-by: Raza Jafri <rjafri@nvidia.com>
|
build |
|
I'm confused by the description here. I don't think this is the right fix. it appears the issue is with classloader and having 2 versions of the DatabricksSparkVersion, I think it needs to be added to the classes we don't shim. We have com/nvidia/spark/rapids/SparkShimVersion and ShimVersion in there already. |
|
You are right, I wanted to put up this patch to unblock the CI last night. I will put up another PR |
This reverts commit 7782695. Signed-off-by: Raza Jafri <rjafri@nvidia.com>
Signed-off-by: Raza Jafri <rjafri@nvidia.com>
|
build |
jlowe
reviewed
Mar 9, 2022
Signed-off-by: Raza Jafri <rjafri@nvidia.com>
|
build |
jlowe
changed the title
jlowe
approved these changes
Mar 9, 2022
tgravescs
approved these changes
Mar 9, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reverting changes to the DB SparkShims as part of the #4857. This change was originally envisioned to avoid referencing the ServiceProvider, but in case of DB we are not reusing the SparkShimImpl so we can directly reference it.Talked to @tgravescs about this and confirmed the correct fix is to "unshim" the
DatabricksShimVersionthat way the class loader will pick up the correct class.fixes #4918
Signed-off-by: Raza Jafri rjafri@nvidia.com