Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] GPU JSON read does not honor ignoreCorruptFiles or ignoreMissingFiles #4803

Closed
jlowe opened this issue Feb 16, 2022 · 0 comments · Fixed by #4809
Closed

[BUG] GPU JSON read does not honor ignoreCorruptFiles or ignoreMissingFiles #4803

jlowe opened this issue Feb 16, 2022 · 0 comments · Fixed by #4809
Assignees
@wbo4958
Labels
bug Something isn't working

Comments

@jlowe
Copy link
Member

jlowe commented Feb 16, 2022

Relates to #4727. If the GPU performs a JSON read operation, it should honor the spark.sql.files.ignoreCorruptFiles and spark.sql.files.ignoreMissingFiles settings.
@jlowe jlowe added bug Something isn't working ? - Needs Triage Need team to review and classify labels Feb 16, 2022
@wbo4958 wbo4958 self-assigned this Feb 17, 2022
@wbo4958 wbo4958 mentioned this issue Feb 17, 2022
Merged
@jlowe jlowe linked a pull request Feb 18, 2022 that will close this issue
Add ignoreCorruptFiles for ORC readers [databricks] #4809
Merged
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Feb 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wbo4958 wbo4958

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add ignoreCorruptFiles for ORC readers [databricks] wbo4958/spark-rapids
3 participants
@wbo4958 @jlowe @sameerz