Add cudf::stable_sort_by_key
#10387
Merged
Add cudf::stable_sort_by_key
#10387
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PointKernel
added
feature request
davidwendt
reviewed
Mar 2, 2022
davidwendt
requested changes
Mar 2, 2022
davidwendt
reviewed
Mar 2, 2022
Codecov Report
@@ Coverage Diff @@
## branch-22.04 #10387 +/- ##
================================================
+ Coverage 10.42% 10.50% +0.07%
================================================
Files 119 126 +7
Lines 20603 21218 +615
================================================
+ Hits 2148 2228 +80
- Misses 18455 18990 +535
Continue to review full report at Codecov.
|
davidwendt
approved these changes
Mar 7, 2022
|
@gpucibot merge |
|
@PointKernel you should be getting two C++ approvals for libcudf PRs before merging. |
Thanks for the reminder. I realized this right after merging. |
rapids-bot bot
pushed a commit
that referenced
this pull request
Mar 12, 2022
Closes #9413 Depending on #10387. There are several changes involved in this PR: - Refactors `cudf::drop_duplicates` to match `std::unique`'s behavior and renames it as `cudf::unique`. `cudf::unique` creates a table by removing duplicate rows in each consecutive group of equivalent rows of the input. - Renames `cudf::unordered_drop_duplicates` as `cudf::distinct`. `cudf::distinct` creates a table by keeping unique rows across the whole input table. Unique rows in the new table are in unspecified orders due to the nature of hash-based algorithms. - Renames `cudf::unordered_distinct_count` as `cudf::distinct_count`: count of `cudf::distinct` - Renames `cudf::distinct_count` as `cudf::unique_count`: count of `cudf::unique` - Updates corresponding tests and benchmarks. - Updates related JNI/Cython bindings. In order not to break the existing behavior in java and python, JNI and Cython bindings of `drop_duplicates` are updated to stably sort the input table first and then `cudf::unique`. Performance hints for `cudf::unique` and `cudf::distinct`: - If the input is pre-sorted, use `cudf::unique` - If the input is **not** pre-sorted and the behavior of `pandas.DataFrame.drop_duplicates` is desired: - If `keep` control (keep the first, last, or none of the duplicates) doesn't matter, use the hash-based `cudf::distinct` - If `keep` control is required, stable sort the input then `cudf::unique` Authors: - Yunsong Wang (https://github.com/PointKernel) Approvers: - Bradley Dice (https://github.com/bdice) - https://github.com/brandon-b-miller - MithunR (https://github.com/mythrocks) - Vyas Ramasubramani (https://github.com/vyasr) URL: #10370
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a new
stable_sort_by_keyAPI into libcudf. The new API is helpful to simplify Cython/JNI bindings ofdrop_duplicates(#10370).