baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

docs: setting n_gqa is no longer required

Browse source
This commit is contained in:
Andrei Betlen 2023-11-22 18:01:54 -05:00
parent 198178225c
commit 68238b7883
1 changed files with 0 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
README.md
View file

@ -143,13 +143,6 @@ For instance, if you want to work with larger contexts, you can expand the conte
llm = Llama(model_path="./models/7B/llama-model.gguf", n_ctx=2048) llm = Llama(model_path="./models/7B/llama-model.gguf", n_ctx=2048)
``` ```
### Loading llama-2 70b
Llama2 70b must set the `n_gqa` parameter (grouped-query attention factor) to 8 when loading:
```python
llm = Llama(model_path="./models/70B/llama-model.gguf", n_gqa=8)
```
## Web Server ## Web Server