Use --ipc=host in docker run for distributed inference (vllm-projec…

…t#1125)
AmazeQiu · Sep 22, 2023 · 7d7e3b7 · 7d7e3b7
1 parent f98b745
commit 7d7e3b7
Showing 1 changed file with 2 additions and 1 deletion.
diff --git a/docs/source/getting_started/installation.rst b/docs/source/getting_started/installation.rst
@@ -46,4 +46,5 @@ You can also build and install vLLM from source:
     .. code-block:: console
 
         $ # Pull the Docker image with CUDA 11.8.
-        $ docker run --gpus all -it --rm --shm-size=8g nvcr.io/nvidia/pytorch:22.12-py3
+        $ # Use `--ipc=host` to make sure the shared memory is large enough.
+        $ docker run --gpus all -it --rm --ipc=host nvcr.io/nvidia/pytorch:22.12-py3