-
Notifications
You must be signed in to change notification settings - Fork 232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tag GpuWindow
child expressions for GPU execution
#6057
Conversation
Fixes NVIDIA#5984. Currently, `UnixTimeExprMeta` postpones part of the initialization of its state to `tagExprForGpu()`. This leads to situations where an incompletely initialized `UnixTimeExprMeta` object is exercised at runtime, such as in NVIDIA#5984. This change moves the initialization to the constructor, leaving `tagExprForGpu()` to deal only with tagging. Signed-off-by: MithunR <mythrocks@gmail.com>
(Tentative fix. Still considering other options to sort out #5984.) |
sql-plugin/src/main/scala/org/apache/spark/sql/rapids/datetimeExpressions.scala
Outdated
Show resolved
Hide resolved
I'll post a better solution to this shortly. |
This reverts commit 0101417. This is the wrong solution. It does not protect against a different, unsupported time-format used in a window function.
This change tags the child expressions of a GpuWindowExecMeta, to ensure that all dependency expressions initialized correctly. Certain child expressions might not be completely initialized even after `GpuOverrides#wrapExpr()` has been called. For instance, `UnixTimeExprMeta` postpones some of its initialization until `tagExprForGpu()` is called. By declaring `windowExpressions`, `partitionSpec`, `orderSpec`, etc. as `childExprs` of `GpuWindowExecMeta`, one guarantees that the `GpuOverrides` infrastructure will tag them for GPU execution. Signed-off-by: MithunR <mythrocks@gmail.com>
GpuWindow
child expressions for GPU execution
The solution has been modified to have |
build |
Thank you for reviewing, @jlowe. I've merged this now. |
Fixes #5984.
This change tags the child expressions of a GpuWindowExecMeta, to ensure
that all dependency expressions initialized correctly.
Certain child expressions might not be completely initialized even after
GpuOverrides#wrapExpr()
has been called. For instance,UnixTimeExprMeta
postpones some of its initialization until
tagExprForGpu()
is called.By declaring
windowExpressions
,partitionSpec
,orderSpec
, etc. aschildExprs
ofGpuWindowExecMeta
, one guarantees that theGpuOverrides
infrastructure will tag them for GPU execution.Signed-off-by: MithunR mythrocks@gmail.com