You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently specifying --enable-cudf-spill or enable_cudf_spill=True enables cuDF spilling on the workers, but not the client which leads to errors like cannot create <class 'cudf.core.buffer.spillable_buffer.SpillableBufferOwner'> without a global spill manager
Since spilling is an option that needs to be constantly set on both the client and workers, enabling this when starting up the workers should also set the same option on the client to avoid both environments being out of sync.
The text was updated successfully, but these errors were encountered:
ayushdg
changed the title
Setting cuDF spilling should also enable it on the client.
Enabling cuDF spilling should also enable it on the client.
Oct 1, 2024
Currently specifying
--enable-cudf-spill
orenable_cudf_spill=True
enables cuDF spilling on the workers, but not the client which leads to errors likecannot create <class 'cudf.core.buffer.spillable_buffer.SpillableBufferOwner'> without a global spill manager
Since spilling is an option that needs to be constantly set on both the client and workers, enabling this when starting up the workers should also set the same option on the client to avoid both environments being out of sync.
cc: @VibhuJawa
The text was updated successfully, but these errors were encountered: