Skip to content

Commit

Permalink
Update docs to better explain limitations of Dataset support (#6010)
Browse files Browse the repository at this point in the history
Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
  • Loading branch information
revans2 authored Jul 19, 2022
1 parent e945e3e commit 94b1263
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions docs/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,11 @@ of focus right now. Other areas like GraphX or RDDs are not accelerated.
### Is the Spark `Dataset` API supported?

The RAPIDS Accelerator supports the `DataFrame` API which is implemented in Spark as `Dataset[Row]`.
If you are using `Dataset[Row]` that is equivalent to the `DataFrame` API. However using custom
classes or types with `Dataset` is not supported. Such queries should still execute correctly when
If you are using `Dataset[Row]` that is equivalent to the `DataFrame` API. In either case the
operations that are supported for acceleration on the GPU are limited. For example using custom
classes or types with `Dataset` are not supported. Neither are using APIs that take `Row` as an input,
or ones that take Scala or Java functions to operate. This includes operators like `flatMap`, `foreach`,
or `foreachPartition`. Such queries will still execute correctly when
using the RAPIDS Accelerator, but it is likely most query operations will not be performed on the
GPU.

Expand Down

0 comments on commit 94b1263

Please sign in to comment.