Merge branch 'main' of github.com:abetlen/llama_cpp_python into main
This commit is contained in:
commit
980903df93
1 changed files with 8 additions and 0 deletions
|
@ -64,12 +64,20 @@ This allows you to use llama.cpp compatible models with any OpenAI compatible cl
|
|||
|
||||
To install the server package and get started:
|
||||
|
||||
Linux/MacOS
|
||||
```bash
|
||||
pip install llama-cpp-python[server]
|
||||
export MODEL=./models/7B/ggml-model.bin
|
||||
python3 -m llama_cpp.server
|
||||
```
|
||||
|
||||
Windows
|
||||
```cmd
|
||||
pip install llama-cpp-python[server]
|
||||
SET MODEL=..\models\7B\ggml-model.bin
|
||||
python3 -m llama_cpp.server
|
||||
```
|
||||
|
||||
Navigate to [http://localhost:8000/docs](http://localhost:8000/docs) to see the OpenAPI documentation.
|
||||
|
||||
## Docker image
|
||||
|
|
Loading…
Reference in a new issue