baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
1219 commits 1 branch 0 tags 2.1 MiB
Commit graph

491 commits

Author SHA1 Message Date
Andrei Betlen
422ebc89ce Fix: Add logit_bias to all completion api methods 2023-11-21 04:01:36 -05:00
Andrei Betlen
07e47f55ba Add support for logit_bias outside of server api. Closes #827 2023-11-21 03:59:46 -05:00
Maarten ter Huurne
c21edb6908
Do not set grammar to None for new LlamaGrammar objects (#834) 
* Do not set `grammar` to `None` for new `LlamaGrammar` objects

The `grammar` attribute is written by `init()`, but that method always
returns `None`, so `__init__()` would then discard the previously
written object.

* Add minimal test for grammar parsing
 2023-11-21 00:23:18 -05:00
mrfakename
ef65fc5ff4
Add MistralLite, Intel, and OpenChat prompt formats (#927) 
* Add MistralLite format

* Update llama_chat_format.py

* Update llama_chat_format.py
 2023-11-21 00:19:25 -05:00
TK-Master
b8438f70b5
Added support for min_p (#921) 
* Added support for min_p

My small contribution to this great project.

Ref: https://github.com/ggerganov/llama.cpp/pull/3841

Closes: https://github.com/abetlen/llama-cpp-python/issues/911

* Fix for negative temp (sample_softmax)
 2023-11-20 23:21:33 -05:00
Andrei Betlen
a34d480141 Fix #929 2023-11-20 22:50:59 -05:00
Andrei Betlen
2c2afa320f Update llama.cpp 2023-11-20 14:11:33 -05:00
Andrei Betlen
f2901d840e Bump version 2023-11-14 14:10:00 -05:00
Andrei Betlen
01846a76b9 Bump version 2023-11-10 16:36:12 -05:00
Andrei Betlen
b7e60b66f4 Bump version 2023-11-10 06:21:24 -05:00
Andrei Betlen
6f0b0b1b84 Fix sampling bug when logits_all=False 2023-11-10 05:15:41 -05:00
Andrei Betlen
d9b38e3e3a Potential bugfix for eval 2023-11-10 04:41:19 -05:00
Andrei Betlen
b84d76a844 Fix: add default stop sequence to chatml chat format 2023-11-10 04:24:48 -05:00
Andrei Betlen
1b376c62b7 Update functionary for new OpenAI API 2023-11-10 02:51:58 -05:00
Andrei Betlen
17da8fb446 Add missing tool_calls finish_reason 2023-11-10 02:51:06 -05:00
Andrei Betlen
770df34436 Add $ref and $defs support to json schema converter 2023-11-10 02:50:46 -05:00
Andrei Betlen
faeae181b1 Fix: json_schema_to_gbnf should take string dump of json schema as input 2023-11-10 02:50:17 -05:00
Andrei Betlen
e7962d2c73 Fix: default max_tokens matches openai api (16 for completion, max length for chat completion) 2023-11-10 02:49:27 -05:00
Andrei Betlen
b62c449839 Bugfix: missing response_format for functionary and llava chat handlers 2023-11-09 00:55:23 -05:00
Andrei Betlen
fd41ed3a90 Add set_seed to Llama class 2023-11-08 11:09:41 -05:00
Andrei Betlen
ca4cb88351 Fix destructor NoneType is not callable error 2023-11-08 11:05:45 -05:00
Andrei Betlen
01cb3a0381 Bump version 2023-11-08 00:54:54 -05:00
Andrei Betlen
b30b9c338b Add JSON mode support. Closes #881 2023-11-08 00:07:16 -05:00
Andrei Betlen
4852a6a39c Fix built in GBNF grammar rules 2023-11-08 00:06:22 -05:00
Andrei Betlen
64f5153c35 Add seed parameter to chat handlers 2023-11-07 23:41:29 -05:00
Andrei Betlen
86aeb9f3a1 Add seed parameter support for completion and chat_completion requests. Closes #884 2023-11-07 23:37:28 -05:00
Damian Stewart
aab74f0b2b
Multimodal Support (Llava 1.5) (#821) 
* llava v1.5 integration

* Point llama.cpp to fork

* Add llava shared library target

* Fix type

* Update llama.cpp

* Add llava api

* Revert changes to llama and llama_cpp

* Update llava example

* Add types for new gpt-4-vision-preview api

* Fix typo

* Update llama.cpp

* Update llama_types to match OpenAI v1 API

* Update ChatCompletionFunction type

* Reorder request parameters

* More API type fixes

* Even More Type Updates

* Add parameter for custom chat_handler to Llama class

* Fix circular import

* Convert to absolute imports

* Fix

* Fix pydantic Jsontype bug

* Accept list of prompt tokens in create_completion

* Add llava1.5 chat handler

* Add Multimodal notebook

* Clean up examples

* Add server docs

---------

Co-authored-by: Andrei Betlen <abetlen@gmail.com>
 2023-11-07 22:48:51 -05:00
Andrei Betlen
56171cf7bf Bump version 2023-11-06 09:37:55 -05:00
Andrei Betlen
be0add1b2d Fix type bug 2023-11-06 09:30:38 -05:00
Andrei Betlen
e214a58422 Refactor Llama class internals 2023-11-06 09:16:36 -05:00
Andrei Betlen
bbffdaebaa Refactor autotokenizer format to reusable function 2023-11-06 09:07:27 -05:00
Joe
4ff8def4d0
#717: Add support for Huggingface Autotokenizer (#790) 
Co-authored-by: Andrei <abetlen@gmail.com>
 2023-11-05 18:06:36 -05:00
earonesty
3580e2c5df
Update llama_chat_format.py (#869) 
* Update llama_chat_format.py

properly formal llama2 with first-message prompt embedded

* Update llama_chat_format.py
 2023-11-05 17:00:13 -05:00
Andrei Betlen
f0b30ef7dc Update llama.cpp 2023-11-05 16:57:10 -05:00
Andrei Betlen
2ec043af76 Clean up stdout / stderr suppression 2023-11-03 13:02:15 -04:00
Andrei Betlen
4ea7027c41 Rename internal only module utils to _utils 2023-11-03 12:55:55 -04:00
Andrei Betlen
df9362eeea Update llama.cpp 2023-11-03 11:34:50 -04:00
Andrei
3af7b21ff1
Add functionary support (#784) 
* Add common grammars and json-schema-to-grammar utility function from llama.cpp

* Pass functions to format function

* Add basic functionary formatting

* Add LlamaChatHandler for more complex chat use cases

* Add function calling example notebook

* Add support for regular chat completions alongside function calling
 2023-11-03 02:12:14 -04:00
Andrei
ab028cb878
Migrate inference to llama_batch and llama_decode api (#795) 
* Add low-level batching notebook

* fix: tokenization of special characters: (#850)

It should behave like llama.cpp, where most out of the box usages
treat special characters accordingly

* Update CHANGELOG

* Cleanup

* Fix runner label

* Update notebook

* Use llama_decode and batch api

* Support logits_all parameter

---------

Co-authored-by: Antoine Lizee <antoine.lizee@gmail.com>
 2023-11-02 20:13:57 -04:00
Andrei Betlen
8350de9a18 Bump version 2023-11-02 15:53:01 -04:00
Andrei Betlen
011b95d7f3 Fix name 'open' is not defined exception. Closes #860 2023-11-02 15:30:55 -04:00
Andrei Betlen
fa83cc5f9c Update llama.cpp 
Fix build examples

Exclude examples directory

Revert cmake changes

Try actions/checkout@v4

Try to update submodules

Revert

Update llama.cpp

Fix build examples

Exclude examples directory

Revert cmake changes

Try actions/checkout@v4

Try to update submodules

Revert
 2023-11-02 14:28:15 -04:00
Antoine Lizee
4d4e0f11e2 fix: tokenization of special characters: (#850) 
It should behave like llama.cpp, where most out of the box usages
treat special characters accordingly
 2023-11-02 14:28:14 -04:00
Andrei Betlen
6b3aa7fc8f Bump version 2023-11-01 19:25:03 -04:00
Sujeendran Menon
7b136bb5b1
Fix for shared library not found and compile issues in Windows (#848) 
* fix windows library dll name issue

* Updated README.md Windows instructions

* Update llama_cpp.py to handle different windows dll file versions
 2023-11-01 18:55:57 -04:00
cebtenzzre
eefd76fe81
llama: fix exception in Llama.__del__ (#846) 2023-11-01 18:53:57 -04:00
David Ponce
3fc9147218
Iterate over tokens that should be biased rather than the entire vocabulary. (#851) 2023-11-01 18:53:47 -04:00
Marko Tasic
9c8f4dca5f
fixed Llama._create_completion suffix check, it can be either None or str instance (#854) 2023-11-01 18:52:50 -04:00
Daniel Thuerck
5f8f369d1b
Pass-Through grammar parameter in web server. (#855) Closes #778 2023-11-01 18:51:12 -04:00
Adam Katora
25cb710281
Update llama_types.py (#849) 
Minor typo fix, funcion -> function
 2023-11-01 18:50:11 -04:00