[auto-merge] branch-0.2 to branch-0.3 - resolve conflict #738
Conversation
Add support to run Pandas UDFs on GPUs, mainly consisting of two things: Overriding all the 6 related plans to build GPU context of device and memory for Python processes. Introducing 2 new python modules rapids.worker and rapids.daemon to execute the GPU memory initialization by leveraging RMM Python APIs. Signed-off-by: Firestarman <firestarmanllc@gmail.com> Co-authored-by: Liangcai Li <liangcail@nvidia.com> Co-authored-by: Robert (Bobby) Evans <bobby@apache.org> Co-authored-by: shotai <gamem.8818@gmail.com>
* Fix collect time in CoalesceBatches * Also move totalTime * switch to use if !isdefined Signed-off-by: Thomas Graves <tgraves@apache.org> * remove extra newline Signed-off-by: Thomas Graves <tgraves@apache.org>
tgravescs
requested review from
GaryShen2008,
jlowe,
NvTimLiu and
revans2
as code owners
|
build |
|
The newly added tests are failing for this in a way that looks like the python files in the jar were not added to the python path. |
|
I'm going to go ahead and merge this. |
|
Note there are new changes on branch-0.2 that are not covered by this. |
|
Also remember to NOT squash this commit, must be a merge commit. |
|
@GaryShen2008 @NvTimLiu @pxLi could you take a look at these test failures, is there something setup different in the 0.3 env? |
@tgravescs nothing different here for 0.3. we are seeing periodically failures should be related recent-merged python udf_test.
|
[auto-merge] branch-0.2 to branch-0.3 - resolve conflict
[auto-merge] branch-0.2 to branch-0.3 - resolve conflict
Fix the conflicts in #736
2 conflicts were jenkins/spark-premerge-build.sh where the profile to include-databricks was removed and then other was in GpuCoalesceBatches where we changed the collect time metrics and the variable hasOnDeck changed names