Merge branch 'main' of github.com:abetlen/llama_cpp_python into main

This commit is contained in:
Andrei Betlen 2023-05-05 15:07:26 -04:00
commit 980903df93

View file

@ -64,12 +64,20 @@ This allows you to use llama.cpp compatible models with any OpenAI compatible cl
To install the server package and get started:
Linux/MacOS
```bash
pip install llama-cpp-python[server]
export MODEL=./models/7B/ggml-model.bin
python3 -m llama_cpp.server
```
Windows
```cmd
pip install llama-cpp-python[server]
SET MODEL=..\models\7B\ggml-model.bin
python3 -m llama_cpp.server
```
Navigate to [http://localhost:8000/docs](http://localhost:8000/docs) to see the OpenAPI documentation.
## Docker image