baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

fix: fix logprobs when BOS is not present (#1471)

Browse source 
* Fix lobprobs when BOS is not present

* Fix logprobs when bos is not available
This commit is contained in:
Asghar Ghorbani 2024-06-04 16:18:38 +02:00 committed by GitHub
parent 027f7bc678
commit 6e0642ca19
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
llama_cpp/llama.py
View file

@ -1410,8 +1410,8 @@ class Llama:
top_logprobs: List[Optional[Dict[str, float]]] = []
if echo:
# Remove leading BOS token
all_tokens = prompt_tokens[1:] + completion_tokens
# Remove leading BOS token if exists
all_tokens = prompt_tokens[1 if prompt_tokens[0] == self.token_bos() else 0:] + completion_tokens
else:
all_tokens = completion_tokens