Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JNI JSON read with DataSource and infered schema, along with basic java nested Schema JSON reads #14954

Merged
merged 6 commits into from
Feb 8, 2024
Merged

JNI JSON read with DataSource and infered schema, along with basic java nested Schema JSON reads #14954

merged 6 commits into from
Feb 8, 2024

Conversation

revans2
Copy link
Contributor

@revans2 revans2 commented Feb 1, 2024

Description

This adds in support for some more JSON reading functionality. It allows us to infer the JSON schema using a DataSource as the input. It also adds in support for using a nested Schema when parsing JSON.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@revans2
JNI JSON read with DataSource and infered schema, along with basic java
da0be29 
support for nested Schema in JSON reads

Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
@revans2 revans2 added 3 - Ready for Review Ready for review by team Java Affects Java cuDF API. Spark Functionality that helps Spark RAPIDS 4 - Needs cuDF (Java) Reviewer improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Feb 1, 2024
@revans2 revans2 self-assigned this Feb 1, 2024
@revans2 revans2 requested a review from a team as a code owner February 1, 2024 20:27
@revans2 revans2 mentioned this pull request Feb 1, 2024
Merged
@GregoryKimball
Copy link
Contributor

GregoryKimball commented Feb 2, 2024 •
edited
Loading

Thank you for working on this! Will these changes close #14830?

@revans2
Copy link
Contributor Author

revans2 commented Feb 2, 2024

Thank you for working on this! Will these changes close #14830?

I don't know totally. @andygrove does this cover what you wanted?

@andygrove
Copy link
Contributor

Thank you for working on this! Will these changes close #14830?

I don't know totally. @andygrove does this cover what you wanted?

Yes, it looks like it does. I just needed the ability to specify a nested schema, and this provides that. Thanks @revans2

@karthikeyann karthikeyann mentioned this pull request Feb 7, 2024
Closed
jlowe
jlowe approved these changes Feb 7, 2024
View reviewed changes
Copy link
Member

@jlowe jlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only nits, lgtm.

java/src/main/java/ai/rapids/cudf/Schema.java Show resolved Hide resolved
java/src/main/java/ai/rapids/cudf/Table.java Outdated Show resolved Hide resolved
java/src/main/java/ai/rapids/cudf/Schema.java Outdated Show resolved Hide resolved
java/src/main/java/ai/rapids/cudf/TableWithMeta.java Outdated Show resolved Hide resolved
java/src/main/native/src/TableJni.cpp Outdated Show resolved Hide resolved
@revans2
Copy link
Contributor Author

revans2 commented Feb 8, 2024

@jlowe please take another look

@revans2
Copy link
Contributor Author

revans2 commented Feb 8, 2024

/merge

@rapids-bot rapids-bot bot merged commit 306c47c into rapidsai:branch-24.04 Feb 8, 2024
68 checks passed
@revans2 revans2 mentioned this pull request Feb 8, 2024
Closed
@andygrove andygrove mentioned this pull request Feb 9, 2024
Closed
@vyasr vyasr added 4 - Needs Review Waiting for reviewer to review or respond and removed 4 - Needs cuDF (Java) Reviewer labels Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jlowe jlowe jlowe approved these changes

Assignees

@revans2 revans2

Labels
3 - Ready for Review Ready for review by team 4 - Needs Review Waiting for reviewer to review or respond improvement Improvement / enhancement to an existing function Java Affects Java cuDF API. non-breaking Non-breaking change Spark Functionality that helps Spark RAPIDS
Projects
cuDF/Dask/Numba/UCX
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@revans2 @GregoryKimball @andygrove @jlowe @vyasr