baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

docs: LLAMA_CUBLAS -> LLAMA_CUDA

Browse source
This commit is contained in:
Andrei Betlen 2024-04-04 02:49:19 -04:00
parent 612e78d322
commit c50309e52a
1 changed files with 3 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
README.md
View file

@ -102,10 +102,10 @@ CMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS" pip install llama-cpp-
<details> <details>
<summary>cuBLAS (CUDA)</summary> <summary>cuBLAS (CUDA)</summary>
To install with cuBLAS, set the `LLAMA_CUBLAS=on` environment variable before installing: To install with cuBLAS, set the `LLAMA_CUDA=on` environment variable before installing:
```bash ```bash
CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install llama-cpp-python CMAKE_ARGS="-DLLAMA_CUDA=on" pip install llama-cpp-python
``` ```
</details> </details>
@ -569,7 +569,7 @@ python3 -m llama_cpp.server --model models/7B/llama-model.gguf
Similar to Hardware Acceleration section above, you can also install with GPU (cuBLAS) support like this: Similar to Hardware Acceleration section above, you can also install with GPU (cuBLAS) support like this:
```bash ```bash
CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install 'llama-cpp-python[server]' CMAKE_ARGS="-DLLAMA_CUDA=on" FORCE_CMAKE=1 pip install 'llama-cpp-python[server]'
python3 -m llama_cpp.server --model models/7B/llama-model.gguf --n_gpu_layers 35 python3 -m llama_cpp.server --model models/7B/llama-model.gguf --n_gpu_layers 35
``` ```