baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

llama_cpp server: add missing top_k param to CreateChatCompletionRequest

Browse source 
`llama.create_chat_completion` definitely has a `top_k` argument, but its missing from `CreateChatCompletionRequest`. decision: add it
This commit is contained in:
Lucas Doyle 2023-04-29 11:52:20 -07:00
parent 1e42913599
commit a5aa6c1478
1 changed files with 1 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
llama_cpp/server/app.py
View file

@ -169,6 +169,7 @@ class CreateChatCompletionRequest(BaseModel):
model: str = model_field model: str = model_field
# llama.cpp specific parameters # llama.cpp specific parameters
top_k: int = 40,
repeat_penalty: float = 1.1 repeat_penalty: float = 1.1
class Config: class Config: