baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Update macOS Metal GPU step 4

Browse source 
* Update "today" to version 0.1.62
* Fix numbering (there were 2 step 4's)
This commit is contained in:
Audrey Roy Greenfeld 2023-07-07 11:15:04 +01:00
parent ca11673061
commit d270ec231a
1 changed files with 4 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
docs/install/macos.md
View file

@ -26,19 +26,19 @@ conda create -n llama python=3.9.16
conda activate llama conda activate llama
``` ```
**(4) Install the LATEST llama-cpp-python.. which, as of just today, happily supports MacOS Metal GPU** **(4) Install the LATEST llama-cpp-python...which happily supports MacOS Metal GPU as of version 0.1.62**
*(you needed xcode installed in order pip to build/compile the C++ code)* *(you needed xcode installed in order pip to build/compile the C++ code)*
``` ```
pip uninstall llama-cpp-python -y pip uninstall llama-cpp-python -y
CMAKE_ARGS="-DLLAMA_METAL=on" FORCE_CMAKE=1 pip install -U llama-cpp-python --no-cache-dir CMAKE_ARGS="-DLLAMA_METAL=on" FORCE_CMAKE=1 pip install -U llama-cpp-python --no-cache-dir
pip install 'llama-cpp-python[server]' pip install 'llama-cpp-python[server]'
# you should now have llama-cpp-python v0.1.62 installed # you should now have llama-cpp-python v0.1.62 or higher installed
llama-cpp-python         0.1.62      llama-cpp-python         0.1.68
``` ```
**(4) Download a v3 ggml model** **(5) Download a v3 ggml model**
- **ggmlv3** - **ggmlv3**
- file name ends with **q4_0.bin** - indicating it is 4bit quantized, with quantisation method 0 - file name ends with **q4_0.bin** - indicating it is 4bit quantized, with quantisation method 0