Jeffrey Morgan
|
a64570dcae
|
Fix clearing kv cache between requests with the same prompt (#2186)
* Fix clearing kv cache between requests with the same prompt
* fix powershell script
|
2024-01-25 13:46:20 -08:00 |
|
Daniel Hiltgen
|
3bc28736cd
|
Merge pull request #2143 from dhiltgen/llm_verbosity
Refine debug logging for llm
|
2024-01-22 13:19:16 -08:00 |
|
Daniel Hiltgen
|
730dcfcc7a
|
Refine debug logging for llm
This wires up logging in llama.cpp to always go to stderr, and also
turns up logging if OLLAMA_DEBUG is set.
|
2024-01-22 12:26:49 -08:00 |
|
Daniel Hiltgen
|
27a2d5af54
|
Debug logging on init failure
|
2024-01-22 12:08:22 -08:00 |
|
Jeffrey Morgan
|
89c4aee29e
|
Unlock mutex when failing to load model (#2117)
|
2024-01-20 20:54:46 -05:00 |
|
Daniel Hiltgen
|
fedd705aea
|
Mechanical switch from log to slog
A few obvious levels were adjusted, but generally everything mapped to "info" level.
|
2024-01-18 14:12:57 -08:00 |
|
Daniel Hiltgen
|
1b249748ab
|
Add multiple CPU variants for Intel Mac
This also refines the build process for the ext_server build.
|
2024-01-17 15:08:54 -08:00 |
|
Bruce MacDonald
|
a897e833b8
|
do not cache prompt (#2018)
- prompt cache causes inferance to hang after some time
|
2024-01-16 13:48:05 -05:00 |
|
Daniel Hiltgen
|
2ecb247276
|
Fix intel mac build
Make sure we're building an x86 ext_server lib when cross-compiling
|
2024-01-13 14:46:34 -08:00 |
|
Daniel Hiltgen
|
39928a42e8
|
Always dynamically load the llm server library
This switches darwin to dynamic loading, and refactors the code now that no
static linking of the library is used on any platform
|
2024-01-11 08:42:47 -08:00 |
|