baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Update changelog

Browse source
This commit is contained in:
Andrei Betlen 2023-07-08 03:37:12 -04:00
parent 24724202ee
commit 670fe4b701
1 changed files with 11 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
CHANGELOG.md
View file

@ -7,30 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
## [Added] ### Added
- (server) Streaming requests can are now interrupted pre-maturely when a concurrent request is made. Can be controlled with the `interrupt_requests` setting. - (server) Streaming requests can are now interrupted pre-maturely when a concurrent request is made. Can be controlled with the `interrupt_requests` setting.
- (server) Moved to fastapi v0.100.0 and pydantic v2
- (docker) Added a new "simple" image that builds llama.cpp from source when started.
## Fixed
- (server) performance improvements by avoiding unnecessary memory allocations during sampling
## [0.1.68] ## [0.1.68]
## [Added] ### Added
- (llama.cpp) Update llama.cpp - (llama.cpp) Update llama.cpp
## [0.1.67] ## [0.1.67]
## Fixed ### Fixed
- Fix performance bug in Llama model by pre-allocating memory tokens and logits. - Fix performance bug in Llama model by pre-allocating memory tokens and logits.
- Fix bug in Llama model where the model was not free'd after use. - Fix bug in Llama model where the model was not free'd after use.
## [0.1.66] ## [0.1.66]
## Added ### Added
- (llama.cpp) New model API - (llama.cpp) New model API
## Fixed ### Fixed
- Performance issue during eval caused by looped np.concatenate call - Performance issue during eval caused by looped np.concatenate call
- State pickling issue when saving cache to disk - State pickling issue when saving cache to disk