Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update array_min and array_max to use new cudf operations #5571

Merged
merged 2 commits into from
May 20, 2022

Conversation

revans2
Copy link
Collaborator

@revans2 revans2 commented May 20, 2022

CUDF finished work on segmented min and segmented max for strings and decimal. This lets us delete a lot of code and hopefully thing run even faster now too.

Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
@revans2 revans2 added the performance A performance related task/issue label May 20, 2022
@revans2 revans2 added this to the May 2 - May 20 milestone May 20, 2022
@revans2 revans2 self-assigned this May 20, 2022
@revans2
Copy link
Collaborator Author

revans2 commented May 20, 2022

build

Copy link
Member

@jlowe jlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Has unused imports that failed the verify build, but otherwise lgtm.

@revans2
Copy link
Collaborator Author

revans2 commented May 20, 2022

build

@revans2
Copy link
Collaborator Author

revans2 commented May 20, 2022

Our build system is too complicated. I do a build/buildall but it does not update the docs or warn me about style. So I have to do a buildall and a regular build for a single version to get everything I need to check before pushing something. This is really frustrating.

Copy link
Collaborator

@gerashegalov gerashegalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gerashegalov
Copy link
Collaborator

Our build system is too complicated. I do a build/buildall but it does not update the docs or warn me about style. So I have to do a buildall and a regular build for a single version to get everything I need to check before pushing something. This is really frustrating.

One can obviously add whatever is missing to buildall . It does scalastyle on the lowest shim involved.

At least now we receive this automated feedback in a couple of minutes instead of waiting for hours: htps://github.com/NVIDIA/spark-rapids/runs/6528205143?check_suite_focus=true

@revans2 revans2 merged commit 3b57770 into NVIDIA:branch-22.06 May 20, 2022
@revans2 revans2 deleted the array_min_max_update branch May 20, 2022 21:05
@revans2
Copy link
Collaborator Author

revans2 commented May 20, 2022

@gerashegalov yes the few mins is so very nice compared to before.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance A performance related task/issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants