Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Failure to recognize ORC file format when loaded via Hive #1216

Closed
jlowe opened this issue Nov 30, 2020 · 0 comments · Fixed by #1220
Closed

[BUG] Failure to recognize ORC file format when loaded via Hive #1216

jlowe opened this issue Nov 30, 2020 · 0 comments · Fixed by #1220
Assignees
@jlowe
Labels
bug Something isn't working P1 Nice to have for release SQL part of the SQL/Dataframe plugin

Comments

@jlowe
Copy link
Member

jlowe commented Nov 30, 2020 •
edited
Loading

Describe the bug
When loading an ORC file from a Hive store the plugin fails to recognize it as a supported file format. Enabling the explain config results in a message like this:

!Exec <FileSourceScanExec> cannot run on GPU because unsupported file format: org.apache.spark.sql.hive.orc.OrcFileFormat

Steps/Code to reproduce bug
Set spark.sql.orc.impl=hive and try to load an ORC file.

Expected behavior
ORC files should be supported even if the Hive implementation was configured.
@jlowe jlowe added bug Something isn't working ? - Needs Triage Need team to review and classify SQL part of the SQL/Dataframe plugin labels Nov 30, 2020
@jlowe jlowe self-assigned this Nov 30, 2020
@jlowe jlowe mentioned this issue Nov 30, 2020
Merged
@jlowe jlowe added P1 Nice to have for release and removed ? - Needs Triage Need team to review and classify labels Nov 30, 2020
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this issue Nov 30, 2023
@nvauto
Update submodule cudf to ceeb39270cac6bbcead175e39482bada45134e24 (NV…
4589b1b 
…IDIA#1216)

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jlowe jlowe

Labels
bug Something isn't working P1 Nice to have for release SQL part of the SQL/Dataframe plugin
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support replacing ORC format when Hive is configured jlowe/spark-rapids
1 participant
@jlowe