Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Add Apache Spark 3.2.0 shim layer #1490

Closed
tgravescs opened this issue Jan 11, 2021 · 2 comments · Fixed by #1704
Closed

[FEA] Add Apache Spark 3.2.0 shim layer #1490

tgravescs opened this issue Jan 11, 2021 · 2 comments · Fixed by #1704
Assignees
@gerashegalov
Labels
P1 Nice to have for release task Work required that improves the product but is not user facing
Milestone
Mar 1 - Mar 12

Comments

@tgravescs
Copy link
Collaborator

Is your feature request related to a problem? Please describe.
Apache spark split the branch for the 3.1.X line which means master branch is now 3.2.0-SNAPSHOT. We need to add a shim layer for this.
@tgravescs tgravescs added feature request New feature or request ? - Needs Triage Need team to review and classify labels Jan 11, 2021
@sameerz sameerz added P1 Nice to have for release and removed ? - Needs Triage Need team to review and classify labels Jan 12, 2021
@sameerz sameerz assigned GaryShen2008 and unassigned GaryShen2008 Jan 12, 2021
@sameerz
Copy link
Collaborator

sameerz commented Jan 12, 2021

Spark 3.2.0 is not required for 0.4, but we should get the shim in place so we can start identifying potential breaking changes.

@sameerz sameerz mentioned this issue Feb 10, 2021
Merged
@sameerz sameerz added this to the Feb 16 - Feb 26 milestone Feb 12, 2021
@sameerz
Copy link
Collaborator

sameerz commented Feb 12, 2021

Support for Spark 3.2.0 will come in the 0.5 version of the plugin.

gerashegalov added a commit that referenced this issue Mar 2, 2021
@gerashegalov
Add a shim provider for Spark 3.2.0 development branch (#1704)
51049a6 
Signed-off-by: Gera Shegalov <gera@apache.org>

Add a shim provider for Spark 3.2.0 development branch. Closes #1490
- fix overflows in aggregate buffer for GpuSum by wiring the explicit output column type
- unit tests for the new shim
- consolidate version profiles in the parent pom
@sameerz sameerz added task Work required that improves the product but is not user facing and removed feature request New feature or request labels Apr 29, 2021
nartal1 pushed a commit to nartal1/spark-rapids that referenced this issue Jun 9, 2021
@gerashegalov
Add a shim provider for Spark 3.2.0 development branch (NVIDIA#1704)
d02ceba 
Signed-off-by: Gera Shegalov <gera@apache.org>

Add a shim provider for Spark 3.2.0 development branch. Closes NVIDIA#1490
- fix overflows in aggregate buffer for GpuSum by wiring the explicit output column type
- unit tests for the new shim
- consolidate version profiles in the parent pom
nartal1 pushed a commit to nartal1/spark-rapids that referenced this issue Jun 9, 2021
@gerashegalov
Add a shim provider for Spark 3.2.0 development branch (NVIDIA#1704)
6614a88 
Signed-off-by: Gera Shegalov <gera@apache.org>

Add a shim provider for Spark 3.2.0 development branch. Closes NVIDIA#1490
- fix overflows in aggregate buffer for GpuSum by wiring the explicit output column type
- unit tests for the new shim
- consolidate version profiles in the parent pom
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this issue Nov 30, 2023
@nvauto
Update submodule cudf to 15baa00693ab4aa59f99ccb417c613880789d047 (NV…
3fb45a6 
…IDIA#1490)

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gerashegalov gerashegalov

Labels
P1 Nice to have for release task Work required that improves the product but is not user facing
Projects
None yet
Milestone
Mar 1 - Mar 12
Development

Successfully merging a pull request may close this issue.

Create a shim for Spark 3.2.0 development gerashegalov/spark-rapids
4 participants
@gerashegalov @tgravescs @sameerz @GaryShen2008