Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename Spark 3.1.0 shim to Spark 3.1.1 to match community #1488

Merged
merged 2 commits into from
Jan 11, 2021

Conversation

tgravescs
Copy link
Collaborator

fixes #1487

Rename the shim to match the community versions. There won't be a 3.1.0 so no reason to keep that shim. I left the 3.1.0-SNAPSHOT as supported in the service provider list for now in case anyone was using those snapshots.

Signed-off-by: Thomas Graves <tgraves@nvidia.com>
Signed-off-by: Thomas Graves <tgraves@nvidia.com>
@tgravescs tgravescs added the feature request New feature or request label Jan 11, 2021
@tgravescs tgravescs added this to the Jan 4 - Jan 15 milestone Jan 11, 2021
@tgravescs tgravescs self-assigned this Jan 11, 2021
@tgravescs
Copy link
Collaborator Author

build

Copy link
Collaborator

@revans2 revans2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only concern is that ParquetMaterializer was moved, but it is not in a package that has the shim name in it, so if it changes again it might cause issues in the future, but this did nothing to cause that issue or fix it so it is fine.

@tgravescs
Copy link
Collaborator Author

@razajafri any reason the ParquetMaterializer wasn't in a shim specific package? if not we should file an issue and put it in one. It can still be in the spark package for example we have: spark311/src/main/scala/org/apache/spark/sql/rapids/shims/spark311/GpuColumnarToRowTransitionExec.scala

@tgravescs tgravescs merged commit 6a445be into NVIDIA:branch-0.4 Jan 11, 2021
@tgravescs tgravescs deleted the spark311 branch January 11, 2021 19:19
@razajafri
Copy link
Collaborator

When I created the file I added a rapids to the end of the package name to make it obvious that it's a rapids file. I can file an issue to add shims.spark311 to the end of the existing package. Will that work?

@tgravescs
Copy link
Collaborator Author

yes please match the other files in the shim with shims.spark311

nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
* Rename spark310 to spark311

Signed-off-by: Thomas Graves <tgraves@nvidia.com>

* Fix typoes

Signed-off-by: Thomas Graves <tgraves@nvidia.com>
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
* Rename spark310 to spark311

Signed-off-by: Thomas Graves <tgraves@nvidia.com>

* Fix typoes

Signed-off-by: Thomas Graves <tgraves@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Change spark 3.1.0 to 3.1.1
4 participants