baalajimaestro/ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
2661 commits 1 branch 0 tags 14 MiB
Commit graph

527 commits

Author SHA1 Message Date
Jeffrey Morgan
a9f6c56652 fix FROM instruction erroring when referring to a file 2023-08-22 09:39:42 -07:00
Ryan Baker
0a892419ad
Strip protocol from model path (#377) 2023-08-21 21:56:56 -07:00
Michael Yang
3b49315f97 retry on unauthorized chunk push 
The token printed for authorized requests has a lifetime of 1h. If an
upload exceeds 1h, a chunk push will fail since the token is created on
a "start upload" request.

This replaces the Pipe with SectionReader which is simpler and
implements Seek, a requirement for makeRequestWithRetry. This is
slightly worse than using a Pipe since the progress update is directly
tied to the chunk size instead of controlled separately.
 2023-08-18 11:23:47 -07:00
Michael Yang
7eda70f23b copy metadata from source 2023-08-17 21:55:25 -07:00
Michael Yang
086449b6c7 fmt 2023-08-17 15:32:31 -07:00
Michael Yang
3cbc6a5c01 fix push manifest 2023-08-17 15:28:12 -07:00
Michael Yang
a894cc792d model and file type as strings 2023-08-17 12:08:04 -07:00
Michael Yang
b963a83559
Merge pull request #364 from jmorganca/chunked-uploads 
reimplement chunked uploads
 2023-08-17 09:58:51 -07:00
Michael Yang
bf6688abe6
Merge pull request #360 from jmorganca/fix-request-copies 
Fix request copies
 2023-08-17 09:58:42 -07:00
Bruce MacDonald
6005b157c2
retry download on network errors 2023-08-17 10:31:45 -04:00
Patrick Devine
14220d9833
set the scopes correctly (#368) 2023-08-16 21:42:02 -07:00
Michael Yang
5dfe91be8b reimplement chunked uploads 2023-08-16 14:50:24 -07:00
Michael Yang
9f944c00f1 push: retry on unauthorized 2023-08-16 11:35:33 -07:00
Michael Yang
56e87cecb1 images: remove body copies 2023-08-16 10:30:41 -07:00
Michael Yang
5d9a4cd251
Merge pull request #348 from jmorganca/cross-repo-mount 
cross repo blob mount
 2023-08-16 09:20:36 -07:00
Bruce MacDonald
1deb35ca64
use loaded llm for generating model file embeddings 2023-08-15 16:12:02 -03:00
Bruce MacDonald
e2de886831
do not regenerate embeddings 2023-08-15 16:10:22 -03:00
Bruce MacDonald
f0d7c2f5ea retry download on network errors 2023-08-15 15:07:19 -03:00
Bruce MacDonald
12052a7624
always remove from in progress map on download 2023-08-15 13:20:32 -03:00
Bruce MacDonald
326de48930 use loaded llm for embeddings 2023-08-15 10:50:54 -03:00
Bruce MacDonald
18f2cb0472 dont log fatal 2023-08-15 10:39:59 -03:00
Michael Yang
e26085b921 close open files 2023-08-14 16:08:06 -07:00
Michael Yang
f594c8eb91 cross repo mount 2023-08-14 15:07:35 -07:00
Bruce MacDonald
f020e1d519 always remove from in progress map on download 2023-08-14 13:09:20 -03:00
Bruce MacDonald
2c8b680b03 use file info for embeddings cache 2023-08-14 12:11:04 -03:00
Bruce MacDonald
99b6b60085 use model bin digest for embed digest 2023-08-14 11:57:12 -03:00
Bruce MacDonald
e9a9580bdd do not regenerate embeddings 
- re-use previously evaluated embeddings when possible
- change embeddings digest identifier to be based on model name and embedded file path
 2023-08-14 10:34:17 -03:00
Patrick Devine
d9cf18e28d
add maximum retries when pushing (#334) 2023-08-11 15:41:55 -07:00
Jeffrey Morgan
1556162c90 create .ollama directory if it doesnt exist 2023-08-11 15:35:55 -07:00
Jeffrey Morgan
148f0225c0 create .ollama directory if it doesnt exist 2023-08-11 15:33:11 -07:00
Michael Yang
6517bcc53c
Merge pull request #290 from jmorganca/add-adapter-layers 
implement loading ggml lora adapters through the modelfile
 2023-08-10 17:23:01 -07:00
Michael Yang
6a6828bddf
Merge pull request #167 from jmorganca/decode-ggml 
partial decode ggml bin for more info
 2023-08-10 17:22:40 -07:00
Patrick Devine
be989d89d1
Token auth (#314) 2023-08-10 11:34:25 -07:00
Jeffrey Morgan
040a5b9750 clean up cli flags 2023-08-10 09:27:03 -07:00
Michael Yang
6de5d032e1 implement loading ggml lora adapters through the modelfile 2023-08-10 09:23:39 -07:00
Michael Yang
fccf8d179f partial decode ggml bin for more info 2023-08-10 09:23:10 -07:00
Bruce MacDonald
4b3507f036 embeddings endpoint 
Co-Authored-By: Jeffrey Morgan <jmorganca@gmail.com>
 2023-08-10 11:45:57 -04:00
Bruce MacDonald
984c9c628c fix embeddings invalid values 2023-08-09 16:50:53 -04:00
Bruce MacDonald
ac971c56d1 Update images.go 2023-08-09 11:31:54 -04:00
Bruce MacDonald
8228d166ce pr comments 2023-08-09 11:31:54 -04:00
Bruce MacDonald
907e6c56b3 unlock downloadu in case or requestDownload err 2023-08-09 11:31:54 -04:00
Bruce MacDonald
868e3b31c7 allow for concurrent pulls of the same files 2023-08-09 11:31:54 -04:00
Bruce MacDonald
09d8bf6730 fix build errors 2023-08-09 10:45:57 -04:00
Bruce MacDonald
7a5f3616fd
embed text document in modelfile 2023-08-09 10:26:19 -04:00
Jeffrey Morgan
cff002b824 use content type application/x-ndjson for streaming responses 2023-08-08 21:38:10 -07:00
Bruce MacDonald
1bee2347be pr feedback 
- defer closing llm on embedding
- do not override licenses
- remove debugging print line
- reformat model file docs
 2023-08-08 17:01:37 -04:00
Jeffrey Morgan
a027a7dd65 add 0.0.0.0 as an allowed origin by default 
Fixes #282
 2023-08-08 13:39:50 -07:00
Bruce MacDonald
884d78ceb3 allow embedding from model binary 2023-08-08 14:38:57 -04:00
Bruce MacDonald
21ddcaa1f1 pr comments 
- default to embeddings enabled
- move embedding logic for loaded model to request
- allow embedding full directory
- close llm on reload
 2023-08-08 13:49:37 -04:00
Michael Yang
f2074ed4c0
Merge pull request #306 from jmorganca/default-keep-system 
automatically set num_keep if num_keep < 0
 2023-08-08 09:25:34 -07:00
Bruce MacDonald
a6f6d18f83 embed text document in modelfile 2023-08-08 11:27:17 -04:00
Bruce MacDonald
34a13a9d05
pass flags to serve to allow setting allowed-origins + host and port 2023-08-08 10:41:42 -04:00
Jeffrey Morgan
8713ac23a8 allow overriding template and system in /api/generate 
Fixes #297
Fixes #296
 2023-08-08 00:55:34 -04:00
Michael Yang
4dc5b117dd automatically set num_keep if num_keep < 0 
num_keep defines how many tokens to keep in the context when truncating
inputs. if left to its default value of -1, the server will calculate
num_keep to be the left of the system instructions
 2023-08-07 16:19:12 -07:00
cmiller01
fb593b7bfc pass flags to serve to allow setting allowed-origins + host and port 
* resolves: https://github.com/jmorganca/ollama/issues/300 and
https://github.com/jmorganca/ollama/issues/282

* example usage:
```
ollama serve --port 9999 --allowed-origins "http://foo.example.com,http://192.0.0.1"
```
 2023-08-07 03:34:37 +00:00
Jeffrey Morgan
e3fb1fd3f1 server: compare options correctly 2023-08-03 15:55:40 -04:00
Michael Yang
a71ff3f6a2 use a pipe to push to registry with progress 
switch to a monolithic upload instead of a chunk upload through a pipe
to report progress
 2023-08-03 10:37:13 -07:00
Bruce MacDonald
8b1e791820
allow specifying zero values in modelfile 2023-08-02 17:07:53 -04:00
Jeffrey Morgan
03cff3a225 server: reset digest at end of generate 2023-08-02 16:15:44 -04:00
Bruce MacDonald
8f8b6288ac
check server is running before running command 2023-08-02 10:51:23 -04:00
Bruce MacDonald
765994362c use head to check heartbeat 2023-08-01 14:50:38 -04:00
Bruce MacDonald
1c5a8770ee read runner parameter options from map 
- read runner options from map to see what was specified explicitly and overwrite zero values
 2023-08-01 13:38:19 -04:00
Bruce MacDonald
daa0d1de7a allow specifying zero values in modelfile 2023-08-01 13:37:50 -04:00
Jeffrey Morgan
528bafa585 cache loaded model 2023-08-01 11:24:18 -04:00
Michael Yang
872011630a fix license 2023-07-31 21:46:48 -07:00
Michael Yang
203fdbc4b8 check err 2023-07-31 21:46:48 -07:00
Michael Yang
70e0ab6b3d remove unnecessary fmt.Sprintf 2023-07-31 21:46:47 -07:00
Jeffrey Morgan
9968153729 fix Go warnings 2023-07-31 21:37:40 -04:00
Bruce MacDonald
671eec6da9 log prediction failures 2023-07-31 16:46:37 -04:00
Michael Yang
eadee46840
Merge pull request #236 from jmorganca/check-os-walk 
check os.Walk err
 2023-07-28 14:14:21 -07:00
Michael Yang
bd58528fbd check os.Walk err 2023-07-28 12:15:31 -07:00
Michael Yang
c5e447a359 remove io/ioutil import 
ioutil is deprecated
 2023-07-28 12:06:03 -07:00
Bruce MacDonald
f5cbcb08e6 specify stop params separately 2023-07-28 11:29:00 -04:00
Bruce MacDonald
184ad8f057 allow specifying stop conditions in modelfile 2023-07-28 11:02:04 -04:00
Bruce MacDonald
0345070dfa
update model file docs 2023-07-28 10:33:52 -04:00
Bruce MacDonald
1ac38ec89c improve modelfile docs 2023-07-27 15:13:04 -04:00
Michael Yang
f62a882760 add session expiration 2023-07-27 09:31:44 -07:00
Michael Yang
32aec66e6a add load duration 2023-07-27 09:31:44 -07:00
Michael Yang
35af37a2cb session id 2023-07-27 09:31:44 -07:00
Bruce MacDonald
4c1caa3733 download models when creating from modelfile 2023-07-25 14:25:13 -04:00
Bruce MacDonald
07ed69bc37 remove reduandant err var 2023-07-25 10:30:14 -04:00
Bruce MacDonald
536028c35a better error message when model not found on pull 2023-07-24 17:48:17 -04:00
Bruce MacDonald
abf614804b
remove file on digest mismatch 2023-07-24 21:59:12 +02:00
Bruce MacDonald
a0dbbb23c4
truncate file size on resume 2023-07-24 21:58:32 +02:00
Bruce MacDonald
0fd6278446 do not panic server if file cannot be opened 2023-07-24 15:24:34 -04:00
Bruce MacDonald
abfc73d31e make response errors unique for error trace 2023-07-24 15:04:21 -04:00
Bruce MacDonald
5a5ca8e7ff remove file on digest mismatch 2023-07-24 14:53:01 -04:00
Bruce MacDonald
fdbef6c95e truncate file size on resume 2023-07-24 14:36:19 -04:00
Patrick Devine
4cb42ca55e
add copy command (#191) 2023-07-24 11:27:28 -04:00
Patrick Devine
88c55199f8
change push to chunked uploads from monolithic (#179) 2023-07-22 17:31:26 -07:00
Michael Yang
8609db77ea use gin-contrib/cors middleware 2023-07-22 09:39:08 -07:00
Patrick Devine
6d6b0d3321
change error handler behavior and fix error when a model isn't found (#173) 2023-07-21 23:02:12 -07:00
Michael Yang
20a5d99f77 fix vars.First 2023-07-21 20:45:32 -07:00
Patrick Devine
3b43cc019a
fix extended tag names (#171) 2023-07-21 20:27:25 -07:00
Patrick Devine
b8421dce3d
get the proper path for blobs to delete (#168) 2023-07-21 17:30:40 -07:00
Patrick Devine
9f6e97865c
allow pushing/pulling to insecure registries (#157) 2023-07-21 15:42:19 -07:00
Bruce MacDonald
7ba1308595
Merge pull request #147 from jmorganca/brucemacd/cli-err-display 
Improve CLI error display
 2023-07-21 16:10:19 +02:00
Patrick Devine
e7a393de54
add rm command for models (#151) 2023-07-20 16:09:23 -07:00
Michael Yang
6cea2061ec windows: fix model pulling 2023-07-20 12:35:04 -07:00
Michael Yang
2832801c2a
Merge pull request #91 from jmorganca/fix-stream-errors 
fix stream errors
 2023-07-20 12:21:59 -07:00
Michael Yang
992892866b
Merge pull request #145 from jmorganca/verify-digest 
verify blob digest
 2023-07-20 12:14:21 -07:00
Michael Yang
1f27d7f1b8 fix stream errors 2023-07-20 12:12:08 -07:00
Michael Yang
bf198c3918 verify blob digest 2023-07-20 11:53:57 -07:00
Bruce MacDonald
09dc6273e3 suppress error when running list before pulling image 2023-07-20 20:53:09 +02:00
Bruce MacDonald
3ec4ebc562 remove unused code 2023-07-20 20:18:00 +02:00
Jeffrey Morgan
d59b164fa2 add prompt back to parser 2023-07-20 01:13:30 -07:00
Jeffrey Morgan
75e508e1d6 remove old templates 2023-07-19 23:47:13 -07:00
Michael Yang
6f046dbf18
Update images.go (#134) 2023-07-19 23:46:01 -07:00
Michael Yang
60b4db6389 add .First 2023-07-19 23:24:32 -07:00
Michael Yang
ca210ba480 handle vnd.ollama.image.prompt for compat 2023-07-19 23:24:32 -07:00
Michael Yang
df146c41e2 separate prompt into template and system 2023-07-19 23:24:31 -07:00
Jeffrey Morgan
2d305fa99a allow relative paths in FROM instruction 2023-07-19 21:55:15 -07:00
Jeffrey Morgan
4ca7c4be1f dont consume reader when calculating digest 2023-07-19 00:47:55 -07:00
Patrick Devine
572fc9099f
add license layers to the parser (#116) 2023-07-18 22:49:38 -07:00
Michael Yang
68df36ae50 fix pull 0 bytes on completed layer 2023-07-18 19:38:11 -07:00
Michael Yang
6e36f948df
Merge pull request #109 from jmorganca/fix-create-memory 
fix memory leak in create
 2023-07-18 17:25:19 -07:00
Michael Yang
553fa39fe8 fix memory leak in create 2023-07-18 17:14:17 -07:00
Patrick Devine
9e15635c2d
attempt two for skipping files in the file walk (#105) 2023-07-18 15:37:01 -07:00
Patrick Devine
9658a5043b
skip files in the list if we can't get the correct model path (#100) 2023-07-18 12:39:08 -07:00
Michael Yang
f20426a768 fix mkdir blob path 2023-07-18 11:24:19 -07:00
Patrick Devine
5bea29f610
add new list command (#97) 2023-07-18 09:09:45 -07:00
Patrick Devine
4a28a2f093
add modelpaths (#96) 2023-07-17 22:44:21 -07:00
Michael Yang
c7dd52271c remove debugging messages 2023-07-17 14:17:34 -07:00
Michael Yang
53d0052c6c unavoid unnecessary type conversion 2023-07-17 12:35:03 -07:00
Michael Yang
28a136e9a3 modelfile params 2023-07-17 12:35:03 -07:00
Michael Yang
3862a51a6a create directories if they do not exist 2023-07-17 11:18:48 -07:00
Michael Yang
bcb612a30a fix file paths for windows 2023-07-17 10:47:47 -07:00
Patrick Devine
2fb52261ad
basic distribution w/ push/pull (#78) 
* basic distribution w/ push/pull

* add the parser

* add create, pull, and push

* changes to the parser, FROM line, and fix commands

* mkdirp new manifest directories

* make `blobs` directory if it does not exist

* fix go warnings

* add progressbar for model pulls

* move model struct

---------

Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>
 2023-07-16 17:02:22 -07:00
Michael Yang
743e957d88 use filepath for os compat 2023-07-14 17:27:14 -07:00
Michael Yang
5ade3db040 fix race 
block on write which only returns when the channel is closed. this is
contrary to the previous arrangement where the handler may return but
the stream hasn't finished writing. it can lead to the client receiving
unexpected responses (since the request has been handled) or worst case
a nil-pointer dereference as the stream tries to flush a nil writer
 2023-07-14 15:10:46 -07:00
Michael Yang
1775647f76 continue conversation 
feed responses back into the llm
 2023-07-13 17:13:00 -07:00
Michael Yang
05e08d2310 return more info in generate response 2023-07-13 09:37:32 -07:00
Michael Yang
31590284a7 fix route 2023-07-12 19:21:49 -07:00
Michael Yang
2666d3c206 fix pull race 2023-07-12 19:07:23 -07:00
Michael Yang
0944b01e7d pull fixes 2023-07-12 09:55:07 -07:00
Michael Yang
a806b03f62 no errgroup 2023-07-11 14:58:10 -07:00
Michael Yang
948323fa78 rename partial file 2023-07-11 13:50:26 -07:00
Michael Yang
e243329e2e check api status 2023-07-11 13:42:05 -07:00
Michael Yang
2a66a1164a common stream producer 2023-07-11 13:42:05 -07:00
Michael Yang
fd4792ec56 call llama.cpp directly from go 2023-07-11 11:59:18 -07:00
Jeffrey Morgan
a3ec1ec2a0 consistent error handling for pull and generate 2023-07-10 21:34:15 -07:00
Michael Yang
edba935d67 return error in generate response 2023-07-10 13:30:10 -07:00
Bruce MacDonald
f5e2e150b8 allow overriding default generate options 2023-07-10 20:58:02 +02:00
Jeffrey Morgan
74e92d1258 add basic / route for server 2023-07-07 23:46:15 -04:00
Bruce MacDonald
f533f85d44 pr feedback 
- move error check to api client pull
- simplify error check in generate
- return nil on any pull error
 2023-07-07 17:12:02 -04:00
Bruce MacDonald
61dd87bd90 if directory cannot be resolved, do not fail 2023-07-07 15:27:43 -04:00
Bruce MacDonald
b24be8c6b3 update directory url 2023-07-07 15:13:41 -04:00
Michael Yang
053739d19f no prompt on empty line 2023-07-07 11:01:44 -07:00
Patrick Devine
3f1b7177f2 pass model and predict options 2023-07-07 09:34:05 -07:00
Michael Yang
b0618a466e generate progress 2023-07-06 17:07:40 -07:00
Michael Yang
c4b9e84945 progress 2023-07-06 17:07:40 -07:00
Michael Yang
15c114decb fix prompt templates 2023-07-06 17:03:18 -07:00
Michael Yang
0637632258 simple pull response 2023-07-06 16:34:44 -04:00
Michael Yang
dd960d1d5e update generate response 2023-07-06 16:34:44 -04:00
Bruce MacDonald
d436d51c78 clean up model pull 2023-07-06 16:34:44 -04:00
Bruce MacDonald
c9f45abef3 resumable downloads 2023-07-06 16:34:44 -04:00
Michael Yang
9b8a456c7d embed templates 2023-07-06 16:34:44 -04:00
Bruce MacDonald
7cf5905063 display pull progress 2023-07-06 16:34:44 -04:00
Michael Yang
580fe8951c free llama model 2023-07-06 16:34:44 -04:00
Michael Yang
68e6b4550c use prompt templates 2023-07-06 16:34:44 -04:00
Bruce MacDonald
a6494f8211 pull models 2023-07-06 16:34:44 -04:00
Michael Yang
1b7183c5a1 enable metal gpu acceleration 
ggml-metal.metal must be in the same directory as the ollama binary
otherwise llama.cpp will not be able to find it and load it.

1. go generate llama/llama_metal.go
2. go build .
3. ./ollama serve
 2023-07-06 16:34:44 -04:00
Jeffrey Morgan
0998d4f0a4 remove debug print statements 2023-07-06 16:34:44 -04:00
Bruce MacDonald
8ea5e5e147 separate routes 2023-07-06 16:34:44 -04:00
Jeffrey Morgan
fd962a36e5 client updates 2023-07-06 16:34:44 -04:00
Jeffrey Morgan
9164981d72 move prompt templates out of python bindings 2023-07-06 16:34:44 -04:00
Jeffrey Morgan
6093a88c1a add llama.cpp go bindings 2023-07-06 16:34:44 -04:00
Jeffrey Morgan
76cb60d496 wip go engine 
Co-authored-by: Patrick Devine <pdevine@sonic.net>
 2023-07-06 16:34:44 -04:00
Jeffrey Morgan
b361fa72ec reorganize directories 2023-06-25 13:08:03 -04:00
Jeffrey Morgan
d3709f85b5 build server into desktop app 2023-06-25 00:30:02 -04:00
Jeffrey Morgan
369108e1ad Add dependencies to requirements.txt 2023-06-24 21:13:26 -04:00
Michael Chiang
5142ba2dad
Update README.md 2023-06-23 22:51:54 -04:00
Bruce MacDonald
c5bafaff54 package server with client 2023-06-23 18:38:22 -04:00
Bruce MacDonald
f0eee3faa0 build server executable 2023-06-23 17:23:30 -04:00
Bruce MacDonald
ebec1c61db load and unload model endpoints 2023-06-23 14:47:57 -04:00
Bruce MacDonald
0758cb2d4b llama server wrapper 2023-06-23 13:10:13 -04:00
Jeffrey Morgan
8fa91332fa initial commit 2023-06-22 18:31:40 -04:00