Refactor Frame reductions #8944
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 61f09eb36445beabc7a01dc065cb9c2666e8ee3c.
…me any/all and clean up Series any/all implementations.
This reverts commit 034036fdc371576d8be1d7e49b753525fe1fc943.
vyasr
added
improvement
|
EDIT 8/4: I had originally posted these benchmarks backwards, my mistake. Benchmarks before Benchmarks after |
@shwina my bad, I had posted the benchmarks backwards :) |
vyasr
added
breaking
kkraus14
reviewed
Aug 4, 2021
kkraus14
approved these changes
Aug 5, 2021
Codecov Report
@@ Coverage Diff @@
## branch-21.10 #8944 +/- ##
===============================================
Coverage ? 10.60%
===============================================
Files ? 116
Lines ? 19003
Branches ? 0
===============================================
Hits ? 2015
Misses ? 16988
Partials ? 0 Continue to review full report at Codecov.
|
kkraus14
approved these changes
Aug 6, 2021
|
hmm, apparently I'm not a python codeowner anymore? |
shwina
reviewed
Aug 6, 2021
|
@shwina I just pushed changes to add tests of index reductions again. Regarding the other change (removing a NotImplementedError), I don't think that's a breaking change because it takes a case that failed before (was marked as unsupported even though it should be supported) and lets it function as expected. |
shwina
approved these changes
Aug 7, 2021
|
@gpucibot merge |
shwina
pushed a commit
to shwina/cudf
that referenced
this pull request
Aug 9, 2021
This PR moves implementations of reductions out of the `Series`/`DataFrame` classes and into `Frame`. The resulting reduction code is implemented in terms of columns, which improves the performance of `DataFrame` reductions, and using a single code path makes it easier to maintain. The `median` and `sum_of_squares` reductions, which were previously only available for `Series`, are now transparently enabled for `DataFrame` as well. This PR also explicitly disables reductions for Index objects to match pandas Index APIs. Since a few reductions had previously been implemented, removing these features constitutes a breaking change. Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Keith Kraus (https://github.com/kkraus14) - Ashwin Srinath (https://github.com/shwina) URL: rapidsai#8944
rapids-bot bot
pushed a commit
that referenced
this pull request
Aug 20, 2021
This PR rewrites DataFrame's `kurtosis` and `skew` to use the `_reduce` method introduced in #8944, and it inlines the logic for the `count` to bypass the `_apply_support_method` machinery. This allows us to remove most of that logic entirely aside from the code for row-wise reductions and scans that dispatches to `cupy`. Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Charles Blackmon-Luca (https://github.com/charlesbluca) URL: #9068
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR moves implementations of reductions out of the
Series/DataFrameclasses and intoFrame. The resulting reduction code is implemented in terms of columns, which improves the performance ofDataFramereductions, and using a single code path makes it easier to maintain. Themedianandsum_of_squaresreductions, which were previously only available forSeries, are now transparently enabled forDataFrameas well.This PR also explicitly disables reductions for Index objects to match pandas Index APIs. Since a few reductions had previously been implemented, removing these features constitutes a breaking change.