Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] config option to enable RMM arena memory resource #864

Closed
JustPlay opened this issue Sep 28, 2020 · 1 comment · Fixed by #871
Closed

[FEA] config option to enable RMM arena memory resource #864

JustPlay opened this issue Sep 28, 2020 · 1 comment · Fixed by #871
Assignees
@rongou
Labels
feature request New feature or request

Comments

@JustPlay
Copy link

JustPlay commented Sep 28, 2020 •
edited
Loading

Is your feature request related to a problem? Please describe.
the pool_memory_resouce in rmm-0.16 has higher OMM and cudaErrorIllegalAddress risk than arena_memory_resource based on my tpc-ds test with PTDS=on

since rmm-0.16 and cudf-0.16 has merged the patch to enable arena, now we need spark-rapids to provide a conf switch

@rongou thanks
@JustPlay JustPlay added ? - Needs Triage Need team to review and classify feature request New feature or request labels Sep 28, 2020
@JustPlay JustPlay changed the title [FEA] spark-rapids to provided a --conf option to enable arena_pool_memory_resouce [FEA] spark-rapids to provided a --conf option to enable arena_memory_resouce Sep 28, 2020
@JustPlay
Copy link
Author

#533

@JustPlay JustPlay changed the title [FEA] spark-rapids to provided a --conf option to enable arena_memory_resouce [FEA] need spark-rapids to provided a --conf option to enable arena_memory_resouce Sep 28, 2020
@rongou rongou mentioned this issue Sep 28, 2020
Merged
@jlowe jlowe changed the title [FEA] need spark-rapids to provided a --conf option to enable arena_memory_resouce [FEA] config option to enable RMM arena memory resource Sep 28, 2020
@jlowe jlowe removed the ? - Needs Triage Need team to review and classify label Sep 28, 2020
@jlowe jlowe assigned jlowe and rongou and unassigned jlowe Sep 28, 2020
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this issue Nov 30, 2023
@nvauto
Update submodule cudf to 33a43ae6eac9d0c8f43ff031db477064e235227d (NV…
ec4a68f 
…IDIA#864)

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>

Signed-off-by: spark-rapids automation <70000568+nvauto@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rongou rongou

Labels
feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add support for using the arena allocator rongou/spark-rapids
3 participants
@rongou @jlowe @JustPlay