Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JNI support for Collect Ops in Reduction #10427

Merged

Conversation

sperlingxx
Copy link
Contributor

Exposes public APIs for collect operations as ReductionAggregation, which are essential to spark-rapids. In addition, this PR also extends the test framework of Reduction to discriminate output types from input types.

Signed-off-by: sperlingxx <lovedreamf@gmail.com>
@sperlingxx sperlingxx added feature request New feature or request 3 - Ready for Review Ready for review by team 4 - Needs cuDF (Java) Reviewer non-breaking Non-breaking change labels Mar 14, 2022
@sperlingxx sperlingxx requested a review from a team as a code owner March 14, 2022 05:46
@github-actions github-actions bot added the Java Affects Java cuDF API. label Mar 14, 2022
@codecov
Copy link

codecov bot commented Mar 14, 2022

Codecov Report

Merging #10427 (0affa7f) into branch-22.04 (b613394) will increase coverage by 0.01%.
The diff coverage is 100.00%.

❗ Current head 0affa7f differs from pull request most recent head d083d4e. Consider uploading reports for the commit d083d4e to get more accurate results

Impacted file tree graph

@@               Coverage Diff                @@
##           branch-22.04   #10427      +/-   ##
================================================
+ Coverage         86.13%   86.15%   +0.01%     
================================================
  Files               139      139              
  Lines             22460    22438      -22     
================================================
- Hits              19347    19332      -15     
+ Misses             3113     3106       -7     
Impacted Files Coverage Δ
python/cudf/cudf/io/parquet.py 92.70% <ø> (ø)
python/cudf/cudf/core/column/column.py 89.16% <100.00%> (+0.17%) ⬆️
python/cudf/cudf/core/dataframe.py 93.57% <100.00%> (ø)
python/cudf/cudf/core/frame.py 91.72% <100.00%> (ø)
python/cudf/cudf/core/groupby/groupby.py 91.57% <100.00%> (+0.08%) ⬆️
python/cudf/cudf/core/single_column_frame.py 97.01% <100.00%> (ø)
python/cudf/cudf/core/column/string.py 88.39% <0.00%> (+0.12%) ⬆️
python/cudf/cudf/core/tools/datetimes.py 84.49% <0.00%> (+0.30%) ⬆️
python/cudf/cudf/core/column/lists.py 90.56% <0.00%> (+0.47%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b613394...d083d4e. Read the comment docs.

Copy link
Member

@jlowe jlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nits but otherwise lgtm.

}

/**
* Merge the partial sets produced by multiple CollectSetAggregations. Each null/nan value will be regarded as
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Merge the partial sets produced by multiple CollectSetAggregations. Each null/nan value will be regarded as
* Merge the partial sets produced by multiple CollectSetAggregations. Each null/NaN value will be regarded as

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

java/src/test/java/ai/rapids/cudf/ReductionTest.java Outdated Show resolved Hide resolved
@sperlingxx
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 4596244 into rapidsai:branch-22.04 Mar 15, 2022
@sperlingxx sperlingxx deleted the collect_ops_reduction_java branch March 15, 2022 05:19
@vyasr vyasr added 4 - Needs Review Waiting for reviewer to review or respond and removed 4 - Needs cuDF (Java) Reviewer labels Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team 4 - Needs Review Waiting for reviewer to review or respond feature request New feature or request Java Affects Java cuDF API. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants