baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

llama_cpp server: move logprobs to supported

Browse source 
I think this is actually supported (its in the arguments of `LLama.__call__`, which is how the completion is invoked). decision: mark as supported
This commit is contained in:
Lucas Doyle 2023-04-29 11:48:49 -07:00
parent b47b9549d5
commit 1e42913599
1 changed files with 1 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
llama_cpp/server/app.py
View file

@ -79,12 +79,11 @@ class CreateCompletionRequest(BaseModel):
echo: bool = False
stop: Optional[List[str]] = []
stream: bool = False
logprobs: Optional[int] = Field(None)
# ignored, but marked as required for the sake of compatibility with openai's api
model: str = model_field
logprobs: Optional[int] = Field(None)
# llama.cpp specific parameters
top_k: int = 40
repeat_penalty: float = 1.1