Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] JSON: Support for basic non-string types #4136

Closed
revans2 opened this issue Nov 17, 2021 · 1 comment
Closed

[FEA] JSON: Support for basic non-string types #4136

revans2 opened this issue Nov 17, 2021 · 1 comment
Assignees
Labels
task Work required that improves the product but is not user facing

Comments

@revans2
Copy link
Collaborator

revans2 commented Nov 17, 2021

After we have basic support for String #4135 we should add in support for other types. This can be broken down further as needed by type.

What we have to worry about is parsing of these types and how java/spark handle parsing compared to how CUDF does it. For some types like boolean, and possibly integers CUDF will probably be fine because it should be defined by the JSON specification. For other types like date/time and Decimal types we might need to read them as Strings and then cast them to a final result. I am not sure about floating point. All of these are things that we need to test and investigate. If we do run into situations where we are not compatible for a given type we might need to add configs, like for CSV, where we can control per type if it is enabled or not.

@revans2 revans2 added ? - Needs Triage Need team to review and classify task Work required that improves the product but is not user facing labels Nov 17, 2021
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Nov 23, 2021
@nartal1
Copy link
Collaborator

nartal1 commented Jan 25, 2022

Experimental JSON reader works for basic non-string types in #4485. Closing.

@nartal1 nartal1 closed this as completed Jan 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
task Work required that improves the product but is not user facing
Projects
None yet
Development

No branches or pull requests

3 participants