Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] To honor spark.rapids.memory.gpu.pool=NONE #1034

Merged
merged 1 commit into from
Oct 28, 2020

Conversation

pxLi
Copy link
Collaborator

@pxLi pxLi commented Oct 28, 2020

Signed-off-by: Peixin Li pxli@nyu.edu

If user set,

--conf spark.rapids.memory.gpu.pooling.enabled=true
--conf spark.rapids.memory.gpu.pool=NONE

The RMM will fail to init w/ CUDA_DEFAULT due to GpuDeviceManager.scala#L191-L192
which assigns non-zero value to adjustedMaxAllocation, and then except at Rmm.java#L179-L182

As spark.rapids.memory.gpu.pooling.enabled will be deprecated, so we make a simple workaround for now

Signed-off-by: Peixin Li <pxli@nyu.edu>
@pxLi pxLi added the bug Something isn't working label Oct 28, 2020
@pxLi
Copy link
Collaborator Author

pxLi commented Oct 28, 2020

build

@jlowe jlowe merged commit 6637b78 into NVIDIA:branch-0.3 Oct 28, 2020
sperlingxx pushed a commit to sperlingxx/spark-rapids that referenced this pull request Nov 20, 2020
Signed-off-by: Peixin Li <pxli@nyu.edu>
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
Signed-off-by: Peixin Li <pxli@nyu.edu>
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
Signed-off-by: Peixin Li <pxli@nyu.edu>
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this pull request Nov 30, 2023
Signed-off-by: Alessandro Bellina <abellina@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants