Skip to content

Commit

Permalink
Mark UVM config as internal until supported (NVIDIA#652)
Browse files Browse the repository at this point in the history
Signed-off-by: Jason Lowe <jlowe@nvidia.com>
  • Loading branch information
jlowe authored Sep 3, 2020
1 parent 6d79d6f commit 21abd9f
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 8 deletions.
1 change: 0 additions & 1 deletion docs/configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ Name | Description | Default Value
<a name="memory.gpu.reserve"></a>spark.rapids.memory.gpu.reserve|The amount of GPU memory that should remain unallocated by RMM and left for system use such as memory needed for kernels, kernel launches or JIT compilation.|1073741824
<a name="memory.host.spillStorageSize"></a>spark.rapids.memory.host.spillStorageSize|Amount of off-heap host memory to use for buffering spilled GPU data before spilling to local disk|1073741824
<a name="memory.pinnedPool.size"></a>spark.rapids.memory.pinnedPool.size|The size of the pinned memory pool in bytes unless otherwise specified. Use 0 to disable the pool.|0
<a name="memory.uvm.enabled"></a>spark.rapids.memory.uvm.enabled|UVM or universal memory can allow main host memory to act essentially as swap for device(GPU) memory. This allows the GPU to process more data than fits in memory, but can result in slower processing. This is an experimental feature.|false
<a name="shuffle.transport.enabled"></a>spark.rapids.shuffle.transport.enabled|When set to true, enable the Rapids Shuffle Transport for accelerated shuffle.|false
<a name="shuffle.transport.maxReceiveInflightBytes"></a>spark.rapids.shuffle.transport.maxReceiveInflightBytes|Maximum aggregate amount of bytes that be fetched at any given time from peers during shuffle|1073741824
<a name="shuffle.ucx.managementServerHost"></a>spark.rapids.shuffle.ucx.managementServerHost|The host to be used to start the management server|null
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -291,13 +291,6 @@ object RapidsConf {
.booleanConf
.createWithDefault(true)

val UVM_ENABLED = conf("spark.rapids.memory.uvm.enabled")
.doc("UVM or universal memory can allow main host memory to act essentially as swap " +
"for device(GPU) memory. This allows the GPU to process more data than fits in memory, but " +
"can result in slower processing. This is an experimental feature.")
.booleanConf
.createWithDefault(false)

val CONCURRENT_GPU_TASKS = conf("spark.rapids.sql.concurrentGpuTasks")
.doc("Set the number of tasks that can execute concurrently per GPU. " +
"Tasks may temporarily block when the number of concurrent tasks in the executor " +
Expand Down Expand Up @@ -334,6 +327,14 @@ object RapidsConf {

// Internal Features

val UVM_ENABLED = conf("spark.rapids.memory.uvm.enabled")
.doc("UVM or universal memory can allow main host memory to act essentially as swap " +
"for device(GPU) memory. This allows the GPU to process more data than fits in memory, but " +
"can result in slower processing. This is an experimental feature.")
.internal()
.booleanConf
.createWithDefault(false)

val EXPORT_COLUMNAR_RDD = conf("spark.rapids.sql.exportColumnarRdd")
.doc("Spark has no simply way to export columnar RDD data. This turns on special " +
"processing/tagging that allows the RDD to be picked back apart into a Columnar RDD.")
Expand Down

0 comments on commit 21abd9f

Please sign in to comment.