Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[REVIEW] Preserve float16 upscaling #9069

Merged
merged 10 commits into from
Aug 26, 2021
Merged

[REVIEW] Preserve float16 upscaling #9069

merged 10 commits into from
Aug 26, 2021

Conversation

galipremsagar
Copy link
Contributor

Fixes: #9065

This PR enables using np.dtype only for __cuda_array_interface__ scenario in as_column. The dtype in this array interface is guaranteed to be numeric which np.dtype can handle. Also there is float16 dtype upcasting logic already inplace below i.e., at line 1760.

@galipremsagar galipremsagar added 3 - Ready for Review Ready for review by team Python Affects Python cuDF API. 4 - Needs cuDF (Python) Reviewer labels Aug 19, 2021
@galipremsagar galipremsagar self-assigned this Aug 19, 2021
@galipremsagar galipremsagar requested a review from a team as a code owner August 19, 2021 00:40
@galipremsagar galipremsagar mentioned this pull request Aug 19, 2021
Closed
@galipremsagar galipremsagar added non-breaking Non-breaking change bug Something isn't working labels Aug 19, 2021
@codecov
Copy link

codecov bot commented Aug 19, 2021 •
edited
Loading

Codecov Report

❗ No coverage uploaded for pull request base (branch-21.10@0ad36ff). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff               @@
##             branch-21.10    #9069   +/-   ##
===============================================
  Coverage                ?   10.76%           
===============================================
  Files                   ?      114           
  Lines                   ?    19086           
  Branches                ?        0           
===============================================
  Hits                    ?     2055           
  Misses                  ?    17031           
  Partials                ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0ad36ff...d3a972c. Read the comment docs.

charlesbluca
charlesbluca reviewed Aug 19, 2021
View reviewed changes
Copy link
Member

@charlesbluca charlesbluca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on @harrism's comment, would we also want to track down what went wrong in cudf._lib.replace.replace_nulls[_scalar]() and add an exception so that issues similar to this fail more gracefully in the future?

If not or if that's better for a separate PR, the current changes LGTM

@vyasr
Copy link
Contributor

vyasr commented Aug 19, 2021

Based on @harrism's comment, would we also want to track down what went wrong in cudf._lib.replace.replace_nulls[_scalar]() and add an exception so that issues similar to this fail more gracefully in the future?

If not or if that's better for a separate PR, the current changes LGTM

I think that's out of scope for this PR, but is definitely important to address. Let's see what others think about the discussion on #9069 before deciding whether to wait a little or move forward immediately on this PR.

shwina
shwina reviewed Aug 25, 2021
View reviewed changes
python/cudf/cudf/core/dtypes.py Outdated
return np.dtype("object")
except TypeError:
pass
else:
if np_dtype.kind not in "biufUOMm":
if np_dtype not in cudf._lib.types.np_to_cudf_types:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make a more explicit constant in types.pyx - SUPPORTED_NUMPY_DTYPES or something of that sort?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, made the change and additional handling along with pytests related to pd/np/arrow inputs.

shwina
shwina approved these changes Aug 25, 2021
View reviewed changes
Copy link
Contributor

@shwina shwina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great - with one minor request!

@galipremsagar
Copy link
Contributor Author

rerun tests

@galipremsagar galipremsagar added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team 4 - Needs cuDF (Python) Reviewer labels Aug 26, 2021
@galipremsagar
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 263190a into rapidsai:branch-21.10 Aug 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@charlesbluca charlesbluca charlesbluca left review comments

@shwina shwina shwina approved these changes

@vyasr vyasr Awaiting requested review from vyasr

Assignees

@galipremsagar galipremsagar

Labels
5 - Ready to Merge Testing and reviews complete, ready to merge bug Something isn't working non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] cudaErrorInvalidValue when creating cudf.Series from float16 CuPy Series
4 participants
@galipremsagar @vyasr @shwina @charlesbluca