vinjn
66ef308abd
Import "containerd/console" lib to support colorful output in Windows terminal
2024-02-15 05:56:45 +00:00
Daniel Hiltgen
29e90cc13b
Implement new Go based Desktop app
...
This focuses on Windows first, but coudl be used for Mac
and possibly linux in the future.
2024-02-15 05:56:45 +00:00
Daniel Hiltgen
ecbfc0182f
Go bump to v1.21 to pick up slog
2024-01-18 14:12:57 -08:00
Daniel Hiltgen
39928a42e8
Always dynamically load the llm server library
...
This switches darwin to dynamic loading, and refactors the code now that no
static linking of the library is used on any platform
2024-01-11 08:42:47 -08:00
Daniel Hiltgen
d4cd695759
Add cgo implementation for llama.cpp
...
Run the server.cpp directly inside the Go runtime via cgo
while retaining the LLM Go abstractions.
2023-12-19 09:05:46 -08:00
Patrick Devine
630518f0d9
Add unit test of API routes ( #1528 )
2023-12-14 16:47:40 -08:00
Michael Yang
7232f1fa41
go mod tidy
2023-12-04 16:59:23 -08:00
Michael Yang
01ea6002c4
replace go-humanize with format.HumanBytes
2023-11-14 14:57:41 -08:00
Michael Yang
341fb7e35f
go mod tidy
2023-11-01 11:54:25 -07:00
Patrick Devine
deeac961bb
new readline library ( #847 )
2023-10-25 16:41:18 -07:00
Ajay Kemparaj
bb8464c0d2
update golang.org/x/net fixes CVE-2023-3978,CVE-2023-39325,CVE-2023-44487 ( #855 )
2023-10-25 16:17:24 -07:00
Bruce MacDonald
a0c3e989de
deprecate modelfile embed command ( #759 )
2023-10-16 11:07:37 -04:00
Michael Yang
8544edca21
parallel chunked downloads
2023-10-06 12:56:43 -07:00
Patrick Devine
87d9efb364
switch to forked readline lib which doesn't wreck the repl prompt ( #578 )
2023-09-22 12:17:45 -07:00
Michael Yang
e9f6df7dca
use slices.DeleteFunc
2023-09-05 09:56:59 -07:00
Bruce MacDonald
42998d797d
subprocess llama.cpp server ( #401 )
...
* remove c code
* pack llama.cpp
* use request context for llama_cpp
* let llama_cpp decide the number of threads to use
* stop llama runner when app stops
* remove sample count and duration metrics
* use go generate to get libraries
* tmp dir for running llm
2023-08-30 16:35:03 -04:00
Michael Yang
d791df75dd
check memory requirements before loading
2023-08-10 09:23:11 -07:00
Bruce MacDonald
a6f6d18f83
embed text document in modelfile
2023-08-08 11:27:17 -04:00
Bruce MacDonald
1c5a8770ee
read runner parameter options from map
...
- read runner options from map to see what was specified explicitly and overwrite zero values
2023-08-01 13:38:19 -04:00
Bruce MacDonald
daa0d1de7a
allow specifying zero values in modelfile
2023-08-01 13:37:50 -04:00
Michael Yang
8609db77ea
use gin-contrib/cors middleware
2023-07-22 09:39:08 -07:00
Patrick Devine
e4d7f3e287
vendor in progress bar and change to bytes instead of bibytes ( #130 )
2023-07-19 17:24:03 -07:00
Michael Yang
84200dcde6
use readline
2023-07-19 13:34:56 -07:00
Patrick Devine
5bea29f610
add new list command ( #97 )
2023-07-18 09:09:45 -07:00
Michael Yang
28a136e9a3
modelfile params
2023-07-17 12:35:03 -07:00
Michael Yang
a806b03f62
no errgroup
2023-07-11 14:58:10 -07:00
Michael Yang
fd4792ec56
call llama.cpp directly from go
2023-07-11 11:59:18 -07:00
Michael Yang
c4b9e84945
progress
2023-07-06 17:07:40 -07:00
Michael Yang
3d6009aae3
run prompts
2023-07-06 17:07:40 -07:00
Bruce MacDonald
7cf5905063
display pull progress
2023-07-06 16:34:44 -04:00
Michael Yang
68e6b4550c
use prompt templates
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
fd962a36e5
client updates
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
6093a88c1a
add llama.cpp go bindings
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
76cb60d496
wip go engine
...
Co-authored-by: Patrick Devine <pdevine@sonic.net>
2023-07-06 16:34:44 -04:00
