Michael Yang
171eb040fc
simplify safetensors reading
2024-05-21 11:28:22 -07:00
Patrick Devine
1e1634daca
update go deps ( #4324 )
2024-05-10 21:39:27 -07:00
Patrick Devine
9f8691c6c8
Add llama2 / torch models for ollama create ( #3607 )
2024-04-15 11:26:42 -07:00
Patrick Devine
2c017ca441
Convert Safetensors to an Ollama model ( #2824 )
2024-03-06 21:01:51 -08:00
Michael Yang
fc483274ad
clean up go.mod
2024-02-23 16:53:36 -08:00
vinjn
66ef308abd
Import "containerd/console" lib to support colorful output in Windows terminal
2024-02-15 05:56:45 +00:00
Daniel Hiltgen
29e90cc13b
Implement new Go based Desktop app
...
This focuses on Windows first, but coudl be used for Mac
and possibly linux in the future.
2024-02-15 05:56:45 +00:00
Daniel Hiltgen
d4cd695759
Add cgo implementation for llama.cpp
...
Run the server.cpp directly inside the Go runtime via cgo
while retaining the LLM Go abstractions.
2023-12-19 09:05:46 -08:00
Michael Yang
7232f1fa41
go mod tidy
2023-12-04 16:59:23 -08:00
Michael Yang
01ea6002c4
replace go-humanize with format.HumanBytes
2023-11-14 14:57:41 -08:00
Michael Yang
341fb7e35f
go mod tidy
2023-11-01 11:54:25 -07:00
Patrick Devine
deeac961bb
new readline library ( #847 )
2023-10-25 16:41:18 -07:00
Ajay Kemparaj
bb8464c0d2
update golang.org/x/net fixes CVE-2023-3978,CVE-2023-39325,CVE-2023-44487 ( #855 )
2023-10-25 16:17:24 -07:00
Bruce MacDonald
a0c3e989de
deprecate modelfile embed command ( #759 )
2023-10-16 11:07:37 -04:00
Michael Yang
8544edca21
parallel chunked downloads
2023-10-06 12:56:43 -07:00
Patrick Devine
c928ceb927
add word wrapping for lines which are longer than the terminal width ( #553 )
2023-09-22 13:36:08 -07:00
Patrick Devine
87d9efb364
switch to forked readline lib which doesn't wreck the repl prompt ( #578 )
2023-09-22 12:17:45 -07:00
Michael Yang
e9f6df7dca
use slices.DeleteFunc
2023-09-05 09:56:59 -07:00
Bruce MacDonald
42998d797d
subprocess llama.cpp server ( #401 )
...
* remove c code
* pack llama.cpp
* use request context for llama_cpp
* let llama_cpp decide the number of threads to use
* stop llama runner when app stops
* remove sample count and duration metrics
* use go generate to get libraries
* tmp dir for running llm
2023-08-30 16:35:03 -04:00
Michael Yang
d791df75dd
check memory requirements before loading
2023-08-10 09:23:11 -07:00
Bruce MacDonald
a6f6d18f83
embed text document in modelfile
2023-08-08 11:27:17 -04:00
Bruce MacDonald
1c5a8770ee
read runner parameter options from map
...
- read runner options from map to see what was specified explicitly and overwrite zero values
2023-08-01 13:38:19 -04:00
Bruce MacDonald
daa0d1de7a
allow specifying zero values in modelfile
2023-08-01 13:37:50 -04:00
Michael Yang
8609db77ea
use gin-contrib/cors middleware
2023-07-22 09:39:08 -07:00
Patrick Devine
e4d7f3e287
vendor in progress bar and change to bytes instead of bibytes ( #130 )
2023-07-19 17:24:03 -07:00
Michael Yang
84200dcde6
use readline
2023-07-19 13:34:56 -07:00
Patrick Devine
5bea29f610
add new list command ( #97 )
2023-07-18 09:09:45 -07:00
Michael Yang
28a136e9a3
modelfile params
2023-07-17 12:35:03 -07:00
Michael Yang
a806b03f62
no errgroup
2023-07-11 14:58:10 -07:00
Michael Yang
fd4792ec56
call llama.cpp directly from go
2023-07-11 11:59:18 -07:00
Michael Yang
c4b9e84945
progress
2023-07-06 17:07:40 -07:00
Michael Yang
3d6009aae3
run prompts
2023-07-06 17:07:40 -07:00
Bruce MacDonald
7cf5905063
display pull progress
2023-07-06 16:34:44 -04:00
Michael Yang
68e6b4550c
use prompt templates
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
fd962a36e5
client updates
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
6093a88c1a
add llama.cpp go bindings
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
76cb60d496
wip go engine
...
Co-authored-by: Patrick Devine <pdevine@sonic.net>
2023-07-06 16:34:44 -04:00
