baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Update context size defaults Close #11

Browse source
This commit is contained in:
Andrei Betlen 2023-04-03 20:11:13 -04:00
parent c0a5c0171f
commit f7ab8d55b2
1 changed files with 1 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
examples/fastapi_server.py
View file

@ -34,6 +34,7 @@ llama = llama_cpp.Llama(
embedding=True, embedding=True,
n_threads=6, n_threads=6,
n_batch=2048, n_batch=2048,
n_ctx=2048,
) )