lower GPU memory reserve to 256MB

[rongou]

The current reserve is set to 1GB, this may be too high causing some queries to run out of memory. Lowering it to a more reasonable value. If a user runs out of memory launching large kernels, they can still tweak this parameter.

Fixes #4045

Signed-off-by: Rong Ou rong.ou@gmail.com

[jlowe]

The reason this was set to a large value originally is because a customer query was casting strings to timestamps, and that's not a rare thing for a query to do. Back then, the string-to-timestamp casting code involved quite a few complicated regex replace with back references, and that kernel had a large thread stack space requirement. So large that the query would not run at all unless the reserve was set to 1GB.

I totally get the desire to lower the amount of reserve memory (it's like getting a bigger GPU for free! 😄), but IMO there's a significant associated risk. The default reserve setting could end up being too low to the point where it would crash on some relatively common query operations. That leads to a very poor first impression, as users would not be told when it crashes that they could try tuning this reserve config or, counterintuitively, lower the amount of memory the plugin memory pool is using to fix the out of memory issue. (This is insufficient memory for the driver to complete a kernel launch rather than insufficient memory for a plugin allocation request.)

The timestamp casting code has changed quite a bit since the reserve was tuned, and there may have been libcudf improvements in the memory utilization of the regex backref kernel. We might be able to get away with a significantly lower setting. However I think we should test the proposed setting given that casting strings to timestamps is probably not a rare occurrence in many ETL pipelines.

[rongou]

For q93, we tried 128MB and 64MB, and both succeeded with 10 runs. Running tests locally on my desktop, it fails with 64MB but passes with 128MB. So I guess we are pretty close to the boundary.

For reference, when I originally worked on the arena allocator, I had to reserve some memory too when the max is not specified, and some testing also led to 64MB: https://github.com/rapidsai/rmm/blob/branch-21.12/include/rmm/mr/device/detail/arena.hpp#L295

I think the reason it worked before was more or less a coincidence, as setting the max pool size to total memory minus a 1GB reserve was not very sensical, but just happened to work.

I'm hoping the changes I'm making to the arena allocator would make it more resistant to memory fragmentation, so maybe we can raise this reserve later.

[rongou]

build

[rongou]

build