Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add in HashPartitioning support for decimal 128 [databricks] #4470

Merged
merged 6 commits into from
Jan 24, 2022

Conversation

nartal1
Copy link
Collaborator

@nartal1 nartal1 commented Jan 6, 2022

This fixes #3883, fixes #3885 and fixes #4410.
This PR depends on cudf PR rapidsai/cudf#9919 .
Added DECIMAL_128 support for the Execs and updated the tests.

Signed-off-by: Niranjan Artal <nartal@nvidia.com>
@nartal1 nartal1 added the task Work required that improves the product but is not user facing label Jan 6, 2022
@nartal1 nartal1 self-assigned this Jan 6, 2022
@nartal1 nartal1 linked an issue Jan 6, 2022 that may be closed by this pull request
@nartal1 nartal1 added this to the Dec 13 - Jan 7 milestone Jan 7, 2022
@nartal1 nartal1 marked this pull request as draft January 7, 2022 02:19
revans2
revans2 previously approved these changes Jan 7, 2022
Copy link
Collaborator

@revans2 revans2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @rwlee and @nartal1

Signed-off-by: Niranjan Artal <nartal@nvidia.com>
Signed-off-by: Niranjan Artal <nartal@nvidia.com>
@nartal1 nartal1 marked this pull request as ready for review January 20, 2022 17:08
@nartal1 nartal1 changed the title WIP: Add in HashPartitioning support for decimal 128 Add in HashPartitioning support for decimal 128 [databricks] Jan 20, 2022
@nartal1
Copy link
Collaborator Author

nartal1 commented Jan 20, 2022

build

@nartal1
Copy link
Collaborator Author

nartal1 commented Jan 20, 2022

CI failed on databricks. Fix is in: #4583 . Will kick off the CI again once that PR is merged.

@nartal1 nartal1 changed the title Add in HashPartitioning support for decimal 128 [databricks] Add in HashPartitioning support for decimal 128 Jan 20, 2022
@nartal1
Copy link
Collaborator Author

nartal1 commented Jan 20, 2022

build

@revans2
Copy link
Collaborator

revans2 commented Jan 21, 2022

The failure looks like it is a bug in our code when trying to create a scalar of a struct with a Decimal in it. It is related, but not 100% directly to this. I'll see if I can reproduce it. Unfortunately, I am not 100% sure we can fix it without some modifications to the CUDF JNI. But we will see.

@revans2
Copy link
Collaborator

revans2 commented Jan 21, 2022

I just filed #4600 for the crash.

@nartal1 nartal1 changed the title Add in HashPartitioning support for decimal 128 Add in HashPartitioning support for decimal 128 [databricks] Jan 21, 2022
@revans2
Copy link
Collaborator

revans2 commented Jan 21, 2022

build

@nartal1
Copy link
Collaborator Author

nartal1 commented Jan 22, 2022

CI passed. @revans2 PTAL.

@revans2 revans2 merged commit 8e7a8c9 into NVIDIA:branch-22.02 Jan 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
task Work required that improves the product but is not user facing
Projects
None yet
2 participants