Skip to content

Commit

Permalink
Revert "Linux nvidia GPU option"
Browse files Browse the repository at this point in the history
This reverts commit 76f0318.
  • Loading branch information
jexp committed Nov 3, 2023
1 parent 76f0318 commit 4071ae1
Showing 1 changed file with 0 additions and 13 deletions.
13 changes: 0 additions & 13 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,19 +38,6 @@ No need to install Ollama manually, it will run in a container as
part of the stack when running with the Linux profile: run `docker compose --profile linux up`.
Make sure to set the `OLLAMA_BASE_URL=http://llm:11434` in the `.env` file when using Ollama docker container.

If run into issues that your Nvidia GPU is not used under Linux (despite using the profile), ensure that you have `nvidia-container-toolkit` installed.
And add this to the `llm` service.

```yaml
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
```

**Windows**
Not supported by Ollama, so Windows users need to generate a OpenAI API key and configure the stack to use `gpt-3.5` or `gpt-4` in the `.env` file.
# Develop
Expand Down

0 comments on commit 4071ae1

Please sign in to comment.