-
Notifications
You must be signed in to change notification settings - Fork 232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rename Spark 3.1.0 shim to Spark 3.1.1 to match community #1488
Conversation
Signed-off-by: Thomas Graves <tgraves@nvidia.com>
Signed-off-by: Thomas Graves <tgraves@nvidia.com>
build |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My only concern is that ParquetMaterializer
was moved, but it is not in a package that has the shim name in it, so if it changes again it might cause issues in the future, but this did nothing to cause that issue or fix it so it is fine.
@razajafri any reason the ParquetMaterializer wasn't in a shim specific package? if not we should file an issue and put it in one. It can still be in the spark package for example we have: spark311/src/main/scala/org/apache/spark/sql/rapids/shims/spark311/GpuColumnarToRowTransitionExec.scala |
When I created the file I added a |
yes please match the other files in the shim with shims.spark311 |
* Rename spark310 to spark311 Signed-off-by: Thomas Graves <tgraves@nvidia.com> * Fix typoes Signed-off-by: Thomas Graves <tgraves@nvidia.com>
* Rename spark310 to spark311 Signed-off-by: Thomas Graves <tgraves@nvidia.com> * Fix typoes Signed-off-by: Thomas Graves <tgraves@nvidia.com>
fixes #1487
Rename the shim to match the community versions. There won't be a 3.1.0 so no reason to keep that shim. I left the 3.1.0-SNAPSHOT as supported in the service provider list for now in case anyone was using those snapshots.