baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

fix: eos/bos_token set correctly for Jinja2ChatFormatter and automatic chat formatter (#1230)

Browse source 
The token strings were not correctly retrieved (empty).
This commit is contained in:
Sigbjørn Skjæret 2024-02-28 07:30:31 +01:00 committed by GitHub
parent fea33c9b94
commit c36ab15e68
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
llama_cpp/llama.py
View file

@ -408,8 +408,8 @@ class Llama:
except:
bos_token_id = self.token_bos()
eos_token = self.detokenize([eos_token_id]).decode("utf-8")
bos_token = self.detokenize([bos_token_id]).decode("utf-8")
eos_token = self._model.token_get_text(eos_token_id)
bos_token = self._model.token_get_text(bos_token_id)
if self.verbose:
print(f"Using chat template: {template}", file=sys.stderr)