Andrei Betlen
baeb7b34b3
Merge branch 'main' of github.com:abetlen/llama_cpp_python into main
2023-11-09 00:55:25 -05:00
Andrei Betlen
b62c449839
Bugfix: missing response_format for functionary and llava chat handlers
2023-11-09 00:55:23 -05:00
Kevin Jung
fb1f956a27
Fix server doc arguments ( #892 )
2023-11-08 23:53:00 -05:00
Andrei Betlen
80f4162bf4
Update llama.cpp
2023-11-08 11:18:15 -05:00
Andrei Betlen
fd41ed3a90
Add set_seed to Llama class
2023-11-08 11:09:41 -05:00
Andrei Betlen
ca4cb88351
Fix destructor NoneType is not callable error
2023-11-08 11:05:45 -05:00
Andrei Betlen
01cb3a0381
Bump version
2023-11-08 00:54:54 -05:00
Andrei Betlen
9ae9c86be0
Update server docs
2023-11-08 00:52:13 -05:00
Andrei Betlen
598780fde8
Update Multimodal notebook
2023-11-08 00:48:25 -05:00
Andrei Betlen
b30b9c338b
Add JSON mode support. Closes #881
2023-11-08 00:07:16 -05:00
Andrei Betlen
4852a6a39c
Fix built in GBNF grammar rules
2023-11-08 00:06:22 -05:00
Andrei Betlen
64f5153c35
Add seed parameter to chat handlers
2023-11-07 23:41:29 -05:00
Andrei Betlen
86aeb9f3a1
Add seed parameter support for completion and chat_completion requests. Closes #884
2023-11-07 23:37:28 -05:00
Andrei Betlen
da1b80285a
Update changelog
2023-11-07 23:15:26 -05:00
Andrei Betlen
9a8e64d29d
Update llama.cpp
2023-11-07 23:14:19 -05:00
Andrei Betlen
3660230faa
Fix docs multi-modal docs
2023-11-07 22:52:08 -05:00
Damian Stewart
aab74f0b2b
Multimodal Support (Llava 1.5) ( #821 )
...
* llava v1.5 integration
* Point llama.cpp to fork
* Add llava shared library target
* Fix type
* Update llama.cpp
* Add llava api
* Revert changes to llama and llama_cpp
* Update llava example
* Add types for new gpt-4-vision-preview api
* Fix typo
* Update llama.cpp
* Update llama_types to match OpenAI v1 API
* Update ChatCompletionFunction type
* Reorder request parameters
* More API type fixes
* Even More Type Updates
* Add parameter for custom chat_handler to Llama class
* Fix circular import
* Convert to absolute imports
* Fix
* Fix pydantic Jsontype bug
* Accept list of prompt tokens in create_completion
* Add llava1.5 chat handler
* Add Multimodal notebook
* Clean up examples
* Add server docs
---------
Co-authored-by: Andrei Betlen <abetlen@gmail.com>
2023-11-07 22:48:51 -05:00
Andrei Betlen
56171cf7bf
Bump version
2023-11-06 09:37:55 -05:00
Andrei Betlen
52320c348c
Add python 3.12 classifier
2023-11-06 09:34:07 -05:00
Andrei Betlen
4286830f16
Add python3.12 tests
2023-11-06 09:32:20 -05:00
Andrei Betlen
be0add1b2d
Fix type bug
2023-11-06 09:30:38 -05:00
Andrei Betlen
e214a58422
Refactor Llama class internals
2023-11-06 09:16:36 -05:00
Andrei Betlen
bbffdaebaa
Refactor autotokenizer format to reusable function
2023-11-06 09:07:27 -05:00
Andrei Betlen
b0e597e46e
Pin python version in release
2023-11-06 08:56:41 -05:00
Joe
4ff8def4d0
#717 : Add support for Huggingface Autotokenizer ( #790 )
...
Co-authored-by: Andrei <abetlen@gmail.com>
2023-11-05 18:06:36 -05:00
earonesty
3580e2c5df
Update llama_chat_format.py ( #869 )
...
* Update llama_chat_format.py
properly formal llama2 with first-message prompt embedded
* Update llama_chat_format.py
2023-11-05 17:00:13 -05:00
Andrei Betlen
f0b30ef7dc
Update llama.cpp
2023-11-05 16:57:10 -05:00
Andrei Betlen
dccbac82eb
Update llama.cpp
2023-11-03 18:12:22 -04:00
Andrei Betlen
2ec043af76
Clean up stdout / stderr suppression
2023-11-03 13:02:15 -04:00
Andrei Betlen
4ea7027c41
Rename internal only module utils to _utils
2023-11-03 12:55:55 -04:00
Andrei Betlen
df9362eeea
Update llama.cpp
2023-11-03 11:34:50 -04:00
Andrei
3af7b21ff1
Add functionary support ( #784 )
...
* Add common grammars and json-schema-to-grammar utility function from llama.cpp
* Pass functions to format function
* Add basic functionary formatting
* Add LlamaChatHandler for more complex chat use cases
* Add function calling example notebook
* Add support for regular chat completions alongside function calling
2023-11-03 02:12:14 -04:00
Andrei Betlen
df31303a12
Update CHANGELOG
2023-11-02 20:16:32 -04:00
Andrei
ab028cb878
Migrate inference to llama_batch and llama_decode api ( #795 )
...
* Add low-level batching notebook
* fix: tokenization of special characters: (#850 )
It should behave like llama.cpp, where most out of the box usages
treat special characters accordingly
* Update CHANGELOG
* Cleanup
* Fix runner label
* Update notebook
* Use llama_decode and batch api
* Support logits_all parameter
---------
Co-authored-by: Antoine Lizee <antoine.lizee@gmail.com>
2023-11-02 20:13:57 -04:00
Andrei Betlen
f436e0c872
Update llama.cpp
2023-11-02 17:34:01 -04:00
Andrei Betlen
8350de9a18
Bump version
2023-11-02 15:53:01 -04:00
Andrei Betlen
9ffe62d665
Update llama.cpp
2023-11-02 15:45:27 -04:00
Andrei Betlen
011b95d7f3
Fix name 'open' is not defined exception. Closes #860
2023-11-02 15:30:55 -04:00
Andrei Betlen
fa83cc5f9c
Update llama.cpp
...
Fix build examples
Exclude examples directory
Revert cmake changes
Try actions/checkout@v4
Try to update submodules
Revert
Update llama.cpp
Fix build examples
Exclude examples directory
Revert cmake changes
Try actions/checkout@v4
Try to update submodules
Revert
2023-11-02 14:28:15 -04:00
Andrei Betlen
ddbd10c442
Fix clblast test
2023-11-02 14:28:15 -04:00
Andrei Betlen
735522272b
Fix runner label
2023-11-02 14:28:15 -04:00
Andrei Betlen
0feffb9c20
Cleanup
2023-11-02 14:28:15 -04:00
Andrei Betlen
7fe0bd3a31
Update CHANGELOG
2023-11-02 14:28:15 -04:00
Antoine Lizee
4d4e0f11e2
fix: tokenization of special characters: ( #850 )
...
It should behave like llama.cpp, where most out of the box usages
treat special characters accordingly
2023-11-02 14:28:14 -04:00
Andrei Betlen
952e4cc3ce
Fix: use linux image for opencl test
2023-11-01 21:31:02 -04:00
Andrei Betlen
8bf7fa6e5f
Add opencl test
2023-11-01 21:18:36 -04:00
Andrei Betlen
446d5f5649
Add metal ci test
2023-11-01 21:15:01 -04:00
Andrei Betlen
c89eadafbf
Update CHANGELOG
2023-11-01 19:40:04 -04:00
Andrei Betlen
6b3aa7fc8f
Bump version
2023-11-01 19:25:03 -04:00
NickAlgra
3fbcded7cd
Add missing n_seq_id to llama_batch ( #842 )
2023-11-01 18:56:29 -04:00
