Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix NullPointerException on null partition insert #1744

Merged
merged 5 commits into from
Mar 2, 2021
Merged

Fix NullPointerException on null partition insert #1744

merged 5 commits into from
Mar 2, 2021

Conversation

gerashegalov
Copy link
Collaborator

Port apache/spark#31320 to close #1735

Signed-off-by: Gera Shegalov gera@apache.org

@gerashegalov
Copy link
Collaborator Author

build

jlowe
jlowe requested changes Feb 17, 2021
View reviewed changes
Copy link
Member

@jlowe jlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1735 is targeting the 0.4 release yet this is targeting branch-0.5.

@gerashegalov
Copy link
Collaborator Author

#1735 is targeting the 0.4 release yet this is targeting branch-0.5.

I thought we merge to the latest branch first and then cherry-pick; changing

@gerashegalov gerashegalov changed the base branch from branch-0.5 to branch-0.4 February 17, 2021 14:53
@gerashegalov
Fix NullPointerException on null partition insert
c51705f 
Closes NVIDIA#1735

Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Copy link
Collaborator Author

build

jlowe
jlowe previously approved these changes Feb 17, 2021
View reviewed changes
@gerashegalov
Copy link
Collaborator Author

gerashegalov commented Feb 17, 2021 •
edited
Loading

👀 into test failures

@gerashegalov
Additional fixes
83ff4eb 
- consistent spark.test.version
- profile for pulling Spark 3.1.1 rc2
- don't error out if the table dropped not there

Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Copy link
Collaborator Author

build

jlowe
jlowe requested changes Feb 18, 2021
View reviewed changes
pom.xml Outdated Show resolved Hide resolved
pom.xml Outdated Show resolved Hide resolved
pom.xml Outdated Show resolved Hide resolved
tests/pom.xml Outdated Show resolved Hide resolved
@gerashegalov gerashegalov mentioned this pull request Feb 19, 2021
Merged
@sameerz sameerz added the Spark 3.1+ Bugs only related to Spark 3.1 or higher label Mar 1, 2021
@jlowe
Copy link
Member

jlowe commented Mar 2, 2021

The 3.1.1 shim has been updated and now Spark 3.1.1 is using the official artifacts. @gerashegalov if you upmerge this PR to the latest on branch-0.4 then it should run with 3.1.1 and the tests should pass.

@gerashegalov
Copy link
Collaborator Author

build

@gerashegalov
Copy link
Collaborator Author

The 3.1.1 shim has been updated and now Spark 3.1.1 is using the official artifacts. @gerashegalov if you upmerge this PR to the latest on branch-0.4 then it should run with 3.1.1 and the tests should pass.

@jlowe
✔️ merged branch-0.4 into the PR branch and mvn -Pspark311tests clean test -Dcuda.version=cuda11 -DwildcardSuites=com.nvidia.spark.rapids.InsertPartition311Suite passes

@gerashegalov gerashegalov merged commit e06c226 into NVIDIA:branch-0.4 Mar 2, 2021
@gerashegalov gerashegalov deleted the issue-1735 branch March 2, 2021 19:02
@gerashegalov gerashegalov added this to the Mar 1 - Mar 12 milestone Mar 2, 2021
@gerashegalov gerashegalov mentioned this pull request Mar 2, 2021
Closed
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
@gerashegalov
Fix NullPointerException on null partition insert (NVIDIA#1744)
f0eeca9 
Port apache/spark#31320 to close NVIDIA#1735

Signed-off-by: Gera Shegalov <gera@apache.org>
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
@gerashegalov
Fix NullPointerException on null partition insert (NVIDIA#1744)
54b2d8f 
Port apache/spark#31320 to close NVIDIA#1735

Signed-off-by: Gera Shegalov <gera@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jlowe jlowe jlowe approved these changes

@GaryShen2008 GaryShen2008 Awaiting requested review from GaryShen2008

@NvTimLiu NvTimLiu Awaiting requested review from NvTimLiu

@revans2 revans2 Awaiting requested review from revans2

@tgravescs tgravescs Awaiting requested review from tgravescs

Assignees
No one assigned
Labels
audit_3.1.0 Spark 3.1+ Bugs only related to Spark 3.1 or higher
Projects
None yet
Milestone
Mar 1 - Mar 12
Development

Successfully merging this pull request may close these issues.

[FEA] Fix NPE for static partition with null in GpuInsertIntoHadoopFsRelationCommand
3 participants
@gerashegalov @jlowe @sameerz