baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

fix: sample idx off-by-one error for logit_processors (#1179)

Browse source 
* fix sample_idx off-by-one error

* self._scores is indexed differently, only modify the index within self._input_ids

---------

Co-authored-by: Andrew Lapp <andrew@rew.la>
Co-authored-by: Andrei <abetlen@gmail.com>
This commit is contained in:
Andrew Lapp 2024-02-13 17:26:07 +00:00 committed by GitHub
parent f7cdf78788
commit d6be5333e1
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
llama_cpp/llama.py
View file

@ -557,7 +557,7 @@ class Llama:
logits[:] = ( logits[:] = (
logits_processor(self._input_ids, logits) logits_processor(self._input_ids, logits)
if idx is None if idx is None
else logits_processor(self._input_ids[:idx], logits) else logits_processor(self._input_ids[:idx + 1], logits)
) )
sampling_params = _LlamaSamplingParams( sampling_params = _LlamaSamplingParams(