Add docs explaining GPU selection env vars
This commit is contained in:
parent
b5fcd9d3aa
commit
b53229a2ed
1 changed files with 10 additions and 0 deletions
10
docs/faq.md
10
docs/faq.md
|
@ -193,3 +193,13 @@ To unload the model and free up memory use:
|
||||||
```shell
|
```shell
|
||||||
curl http://localhost:11434/api/generate -d '{"model": "llama2", "keep_alive": 0}'
|
curl http://localhost:11434/api/generate -d '{"model": "llama2", "keep_alive": 0}'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Controlling which GPUs to use
|
||||||
|
|
||||||
|
By default, on Linux and Windows, Ollama will attempt to use Nvidia GPUs, or
|
||||||
|
Radeon GPUs, and will use all the GPUs it can find. You can limit which GPUs
|
||||||
|
will be utilized by setting the environment variable `CUDA_VISIBLE_DEVICES` for
|
||||||
|
NVIDIA cards, or `HIP_VISIBLE_DEVICES` for Radeon GPUs to a comma delimited list
|
||||||
|
of GPU IDs. You can see the list of devices with GPU tools such as `nvidia-smi` or
|
||||||
|
`rocminfo`. You can set to an invalid GPU ID (e.g., "-1") to bypass the GPU and
|
||||||
|
fallback to CPU.
|
Loading…
Reference in a new issue