baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

docs: Update readme examples to use newer Qwen2 model (#1544)

Browse source
This commit is contained in:
Jon Craton 2024-06-21 12:10:15 -04:00 committed by GitHub
parent 5beec1a1fd
commit 27d53589ff
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
README.md
View file

@ -327,7 +327,7 @@ You'll need to install the `huggingface-hub` package to use this feature (`pip i
```python ```python
llm = Llama.from_pretrained( llm = Llama.from_pretrained(
repo_id="Qwen/Qwen1.5-0.5B-Chat-GGUF", repo_id="Qwen/Qwen2-0.5B-Instruct-GGUF",
filename="*q8_0.gguf", filename="*q8_0.gguf",
verbose=False verbose=False
) )
@ -688,7 +688,7 @@ For possible options, see [llama_cpp/llama_chat_format.py](llama_cpp/llama_chat_
If you have `huggingface-hub` installed, you can also use the `--hf_model_repo_id` flag to load a model from the Hugging Face Hub. If you have `huggingface-hub` installed, you can also use the `--hf_model_repo_id` flag to load a model from the Hugging Face Hub.
```bash ```bash
python3 -m llama_cpp.server --hf_model_repo_id Qwen/Qwen1.5-0.5B-Chat-GGUF --model '*q8_0.gguf' python3 -m llama_cpp.server --hf_model_repo_id Qwen/Qwen2-0.5B-Instruct-GGUF --model '*q8_0.gguf'
``` ```
### Web Server Features ### Web Server Features