baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
1141 commits 1 branch 0 tags 2.1 MiB
Commit graph

4 commits

Author SHA1 Message Date
Andrei
ab028cb878
Migrate inference to llama_batch and llama_decode api (#795) 
* Add low-level batching notebook

* fix: tokenization of special characters: (#850)

It should behave like llama.cpp, where most out of the box usages
treat special characters accordingly

* Update CHANGELOG

* Cleanup

* Fix runner label

* Update notebook

* Use llama_decode and batch api

* Support logits_all parameter

---------

Co-authored-by: Antoine Lizee <antoine.lizee@gmail.com>
 2023-11-02 20:13:57 -04:00
Andrei Betlen
c7788c85ab Add Guidance example 2023-05-19 03:16:58 -04:00
Andrei Betlen
6d1bda443e Add clients example. Closes #46 2023-04-08 09:35:32 -04:00
Andrei Betlen
c16bda5fb9 Add performance tuning notebook 2023-04-05 04:09:19 -04:00