[FEA] Replace whole-buffer slicing with reference count increment #585
Labels
feature request
New feature or request
good first issue
Good for newcomers
P2
Not required for release
performance
A performance related task/issue
There are a number of places in the plugin where we slice an entire memory buffer, i.e.:
.slice(0, buffer.getLength())
, to increment the reference count on the buffer. Every slice of a memory buffer creates two objects, the sliced buffer result and a sliced buffer cleaner.This can add up when slicing during compressed shuffle, where each compressed buffer is whole-sliced by every column in every partition essentially just to increment the reference count. cudf recently added
incRefCount
to memory buffers, and it would create a lot less object garbage on the heap to callincRefCount
on the buffer directly rather than slicing the whole buffer.The text was updated successfully, but these errors were encountered: