baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Update README.md

Browse source 
add windows server commad
This commit is contained in:
Thomas Neu 2023-05-05 14:21:57 +02:00 committed by GitHub
parent 5f583b0179
commit 952ba9ecaf
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 8 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
README.md
View file

@ -64,12 +64,20 @@ This allows you to use llama.cpp compatible models with any OpenAI compatible cl
To install the server package and get started:
Linux
```bash
pip install llama-cpp-python[server]
export MODEL=./models/7B/ggml-model.bin
python3 -m llama_cpp.server
```
Windows
```cmd
pip install llama-cpp-python[server]
SET MODEL=\models\7B\ggml-model.bin
python3 -m llama_cpp.server
```
Navigate to [http://localhost:8000/docs](http://localhost:8000/docs) to see the OpenAPI documentation.
## Docker image