Bruce MacDonald
09dd2aeff9
GGUF support ( #441 )
2023-09-07 13:55:37 -04:00
Michael Yang
83c6be1666
fix model manifests ( #477 )
2023-09-06 17:30:08 -04:00
Patrick Devine
790d24eb7b
add show command ( #474 )
2023-09-06 11:04:17 -07:00
Michael Yang
a1ecdd36d5
create manifests directory
2023-09-05 17:10:40 -07:00
Michael Yang
d1c2558f7e
Merge pull request #461 from jmorganca/mxyng/fix-inherit-params
...
fix inherit params
2023-09-05 12:30:23 -07:00
Michael Yang
06ef90c051
fix parameter inheritence
...
parameters are not inherited because they are processed differently from
other layer. fix this by explicitly merging the inherited params into
the new params. parameter values defined in the new modelfile will
override those defined in the inherited modelfile. array lists are
replaced instead of appended
2023-09-05 11:40:20 -07:00
Michael Yang
e9f6df7dca
use slices.DeleteFunc
2023-09-05 09:56:59 -07:00
Michael Yang
681f3c4c42
fix num_keep
2023-09-03 17:47:49 -04:00
Quinn Slack
62d29b2157
do not HTML-escape prompt
...
The `html/template` package automatically HTML-escapes interpolated strings in templates. This behavior is undesirable because it causes prompts like `<h1>hello` to be escaped to `<h1>hello` before being passed to the LLM.
The included test case passes, but before the code change, it failed:
```
--- FAIL: TestModelPrompt
images_test.go:21: got "a<h1>b", want "a<h1>b"
```
2023-09-01 17:16:38 -05:00
Michael Yang
1c8fd627ad
windows: fix create modelfile
2023-08-31 09:47:10 -04:00
Michael Yang
ae950b00f1
windows: fix delete
2023-08-31 09:47:10 -04:00
Michael Yang
eeb40a672c
fix list models for windows
2023-08-31 09:47:10 -04:00
Michael Yang
0f541a0367
s/ListResponseModel/ModelResponse/
2023-08-31 09:47:10 -04:00
Bruce MacDonald
42998d797d
subprocess llama.cpp server ( #401 )
...
* remove c code
* pack llama.cpp
* use request context for llama_cpp
* let llama_cpp decide the number of threads to use
* stop llama runner when app stops
* remove sample count and duration metrics
* use go generate to get libraries
* tmp dir for running llm
2023-08-30 16:35:03 -04:00
Quinn Slack
f4432e1dba
treat stop as stop sequences, not exact tokens ( #442 )
...
The `stop` option to the generate API is a list of sequences that should cause generation to stop. Although these are commonly called "stop tokens", they do not necessarily correspond to LLM tokens (per the LLM's tokenizer). For example, if the caller sends a generate request with `"stop":["\n"]`, then generation should stop on any token containing `\n` (and trim `\n` from the output), not just if the token exactly matches `\n`. If `stop` were interpreted strictly as LLM tokens, then it would require callers of the generate API to know the LLM's tokenizer and enumerate many tokens in the `stop` list.
Fixes https://github.com/jmorganca/ollama/issues/295 .
2023-08-30 11:53:42 -04:00
Michael Yang
982c535428
Merge pull request #428 from jmorganca/mxyng/upload-chunks
...
update upload chunks
2023-08-30 07:47:17 -07:00
Patrick Devine
8bbff2df98
add model IDs ( #439 )
2023-08-28 20:50:24 -07:00
Michael Yang
16b06699fd
remove unused parameter
2023-08-28 18:35:18 -04:00
Michael Yang
246dc65417
loosen http status code checks
2023-08-28 18:34:53 -04:00
Michael Yang
865fceb73c
chunked pipe
2023-08-28 18:34:53 -04:00
Michael Yang
72266c7684
bump chunk size to 95MB
2023-08-28 18:34:53 -04:00
Michael Yang
59734ca24d
set default template
2023-08-26 12:20:48 -07:00
Michael Yang
32d1a00017
remove unused requestContextKey
2023-08-22 10:49:54 -07:00
Michael Yang
04e2128273
move upload funcs to upload.go
2023-08-22 10:49:53 -07:00
Michael Yang
2cc634689b
use url.URL
2023-08-22 10:49:07 -07:00
Michael Yang
95187d7e1e
build release mode
2023-08-22 09:52:43 -07:00
Michael Yang
9ec7e37534
Merge pull request #392 from jmorganca/mxyng/version
...
add version
2023-08-22 09:50:25 -07:00
Michael Yang
2c7f956b38
add version
2023-08-22 09:40:58 -07:00
Jeffrey Morgan
a9f6c56652
fix FROM
instruction erroring when referring to a file
2023-08-22 09:39:42 -07:00
Ryan Baker
0a892419ad
Strip protocol from model path ( #377 )
2023-08-21 21:56:56 -07:00
Michael Yang
3b49315f97
retry on unauthorized chunk push
...
The token printed for authorized requests has a lifetime of 1h. If an
upload exceeds 1h, a chunk push will fail since the token is created on
a "start upload" request.
This replaces the Pipe with SectionReader which is simpler and
implements Seek, a requirement for makeRequestWithRetry. This is
slightly worse than using a Pipe since the progress update is directly
tied to the chunk size instead of controlled separately.
2023-08-18 11:23:47 -07:00
Michael Yang
7eda70f23b
copy metadata from source
2023-08-17 21:55:25 -07:00
Michael Yang
086449b6c7
fmt
2023-08-17 15:32:31 -07:00
Michael Yang
3cbc6a5c01
fix push manifest
2023-08-17 15:28:12 -07:00
Michael Yang
a894cc792d
model and file type as strings
2023-08-17 12:08:04 -07:00
Michael Yang
b963a83559
Merge pull request #364 from jmorganca/chunked-uploads
...
reimplement chunked uploads
2023-08-17 09:58:51 -07:00
Michael Yang
bf6688abe6
Merge pull request #360 from jmorganca/fix-request-copies
...
Fix request copies
2023-08-17 09:58:42 -07:00
Bruce MacDonald
6005b157c2
retry download on network errors
2023-08-17 10:31:45 -04:00
Patrick Devine
14220d9833
set the scopes correctly ( #368 )
2023-08-16 21:42:02 -07:00
Michael Yang
5dfe91be8b
reimplement chunked uploads
2023-08-16 14:50:24 -07:00
Michael Yang
9f944c00f1
push: retry on unauthorized
2023-08-16 11:35:33 -07:00
Michael Yang
56e87cecb1
images: remove body copies
2023-08-16 10:30:41 -07:00
Michael Yang
5d9a4cd251
Merge pull request #348 from jmorganca/cross-repo-mount
...
cross repo blob mount
2023-08-16 09:20:36 -07:00
Bruce MacDonald
1deb35ca64
use loaded llm for generating model file embeddings
2023-08-15 16:12:02 -03:00
Bruce MacDonald
e2de886831
do not regenerate embeddings
2023-08-15 16:10:22 -03:00
Bruce MacDonald
f0d7c2f5ea
retry download on network errors
2023-08-15 15:07:19 -03:00
Bruce MacDonald
12052a7624
always remove from in progress map on download
2023-08-15 13:20:32 -03:00
Bruce MacDonald
326de48930
use loaded llm for embeddings
2023-08-15 10:50:54 -03:00
Bruce MacDonald
18f2cb0472
dont log fatal
2023-08-15 10:39:59 -03:00
Michael Yang
e26085b921
close open files
2023-08-14 16:08:06 -07:00
Michael Yang
f594c8eb91
cross repo mount
2023-08-14 15:07:35 -07:00
Bruce MacDonald
f020e1d519
always remove from in progress map on download
2023-08-14 13:09:20 -03:00
Bruce MacDonald
2c8b680b03
use file info for embeddings cache
2023-08-14 12:11:04 -03:00
Bruce MacDonald
99b6b60085
use model bin digest for embed digest
2023-08-14 11:57:12 -03:00
Bruce MacDonald
e9a9580bdd
do not regenerate embeddings
...
- re-use previously evaluated embeddings when possible
- change embeddings digest identifier to be based on model name and embedded file path
2023-08-14 10:34:17 -03:00
Patrick Devine
d9cf18e28d
add maximum retries when pushing ( #334 )
2023-08-11 15:41:55 -07:00
Jeffrey Morgan
1556162c90
create .ollama
directory if it doesnt exist
2023-08-11 15:35:55 -07:00
Jeffrey Morgan
148f0225c0
create .ollama
directory if it doesnt exist
2023-08-11 15:33:11 -07:00
Michael Yang
6517bcc53c
Merge pull request #290 from jmorganca/add-adapter-layers
...
implement loading ggml lora adapters through the modelfile
2023-08-10 17:23:01 -07:00
Michael Yang
6a6828bddf
Merge pull request #167 from jmorganca/decode-ggml
...
partial decode ggml bin for more info
2023-08-10 17:22:40 -07:00
Patrick Devine
be989d89d1
Token auth ( #314 )
2023-08-10 11:34:25 -07:00
Jeffrey Morgan
040a5b9750
clean up cli flags
2023-08-10 09:27:03 -07:00
Michael Yang
6de5d032e1
implement loading ggml lora adapters through the modelfile
2023-08-10 09:23:39 -07:00
Michael Yang
fccf8d179f
partial decode ggml bin for more info
2023-08-10 09:23:10 -07:00
Bruce MacDonald
4b3507f036
embeddings endpoint
...
Co-Authored-By: Jeffrey Morgan <jmorganca@gmail.com>
2023-08-10 11:45:57 -04:00
Bruce MacDonald
984c9c628c
fix embeddings invalid values
2023-08-09 16:50:53 -04:00
Bruce MacDonald
ac971c56d1
Update images.go
2023-08-09 11:31:54 -04:00
Bruce MacDonald
8228d166ce
pr comments
2023-08-09 11:31:54 -04:00
Bruce MacDonald
907e6c56b3
unlock downloadu in case or requestDownload err
2023-08-09 11:31:54 -04:00
Bruce MacDonald
868e3b31c7
allow for concurrent pulls of the same files
2023-08-09 11:31:54 -04:00
Bruce MacDonald
09d8bf6730
fix build errors
2023-08-09 10:45:57 -04:00
Bruce MacDonald
7a5f3616fd
embed text document in modelfile
2023-08-09 10:26:19 -04:00
Jeffrey Morgan
cff002b824
use content type application/x-ndjson
for streaming responses
2023-08-08 21:38:10 -07:00
Bruce MacDonald
1bee2347be
pr feedback
...
- defer closing llm on embedding
- do not override licenses
- remove debugging print line
- reformat model file docs
2023-08-08 17:01:37 -04:00
Jeffrey Morgan
a027a7dd65
add 0.0.0.0
as an allowed origin by default
...
Fixes #282
2023-08-08 13:39:50 -07:00
Bruce MacDonald
884d78ceb3
allow embedding from model binary
2023-08-08 14:38:57 -04:00
Bruce MacDonald
21ddcaa1f1
pr comments
...
- default to embeddings enabled
- move embedding logic for loaded model to request
- allow embedding full directory
- close llm on reload
2023-08-08 13:49:37 -04:00
Michael Yang
f2074ed4c0
Merge pull request #306 from jmorganca/default-keep-system
...
automatically set num_keep if num_keep < 0
2023-08-08 09:25:34 -07:00
Bruce MacDonald
a6f6d18f83
embed text document in modelfile
2023-08-08 11:27:17 -04:00
Bruce MacDonald
34a13a9d05
pass flags to serve
to allow setting allowed-origins + host and port
2023-08-08 10:41:42 -04:00
Jeffrey Morgan
8713ac23a8
allow overriding template
and system
in /api/generate
...
Fixes #297
Fixes #296
2023-08-08 00:55:34 -04:00
Michael Yang
4dc5b117dd
automatically set num_keep if num_keep < 0
...
num_keep defines how many tokens to keep in the context when truncating
inputs. if left to its default value of -1, the server will calculate
num_keep to be the left of the system instructions
2023-08-07 16:19:12 -07:00
cmiller01
fb593b7bfc
pass flags to serve
to allow setting allowed-origins + host and port
...
* resolves: https://github.com/jmorganca/ollama/issues/300 and
https://github.com/jmorganca/ollama/issues/282
* example usage:
```
ollama serve --port 9999 --allowed-origins "http://foo.example.com,http://192.0.0.1 "
```
2023-08-07 03:34:37 +00:00
Jeffrey Morgan
e3fb1fd3f1
server: compare options correctly
2023-08-03 15:55:40 -04:00
Michael Yang
a71ff3f6a2
use a pipe to push to registry with progress
...
switch to a monolithic upload instead of a chunk upload through a pipe
to report progress
2023-08-03 10:37:13 -07:00
Bruce MacDonald
8b1e791820
allow specifying zero values in modelfile
2023-08-02 17:07:53 -04:00
Jeffrey Morgan
03cff3a225
server: reset digest at end of generate
2023-08-02 16:15:44 -04:00
Bruce MacDonald
8f8b6288ac
check server is running before running command
2023-08-02 10:51:23 -04:00
Bruce MacDonald
765994362c
use head to check heartbeat
2023-08-01 14:50:38 -04:00
Bruce MacDonald
1c5a8770ee
read runner parameter options from map
...
- read runner options from map to see what was specified explicitly and overwrite zero values
2023-08-01 13:38:19 -04:00
Bruce MacDonald
daa0d1de7a
allow specifying zero values in modelfile
2023-08-01 13:37:50 -04:00
Jeffrey Morgan
528bafa585
cache loaded model
2023-08-01 11:24:18 -04:00
Michael Yang
872011630a
fix license
2023-07-31 21:46:48 -07:00
Michael Yang
203fdbc4b8
check err
2023-07-31 21:46:48 -07:00
Michael Yang
70e0ab6b3d
remove unnecessary fmt.Sprintf
2023-07-31 21:46:47 -07:00
Jeffrey Morgan
9968153729
fix Go warnings
2023-07-31 21:37:40 -04:00
Bruce MacDonald
671eec6da9
log prediction failures
2023-07-31 16:46:37 -04:00
Michael Yang
eadee46840
Merge pull request #236 from jmorganca/check-os-walk
...
check os.Walk err
2023-07-28 14:14:21 -07:00
Michael Yang
bd58528fbd
check os.Walk err
2023-07-28 12:15:31 -07:00
Michael Yang
c5e447a359
remove io/ioutil import
...
ioutil is deprecated
2023-07-28 12:06:03 -07:00
Bruce MacDonald
f5cbcb08e6
specify stop params separately
2023-07-28 11:29:00 -04:00
Bruce MacDonald
184ad8f057
allow specifying stop conditions in modelfile
2023-07-28 11:02:04 -04:00
Bruce MacDonald
0345070dfa
update model file docs
2023-07-28 10:33:52 -04:00
Bruce MacDonald
1ac38ec89c
improve modelfile docs
2023-07-27 15:13:04 -04:00
Michael Yang
f62a882760
add session expiration
2023-07-27 09:31:44 -07:00
Michael Yang
32aec66e6a
add load duration
2023-07-27 09:31:44 -07:00
Michael Yang
35af37a2cb
session id
2023-07-27 09:31:44 -07:00
Bruce MacDonald
4c1caa3733
download models when creating from modelfile
2023-07-25 14:25:13 -04:00
Bruce MacDonald
07ed69bc37
remove reduandant err var
2023-07-25 10:30:14 -04:00
Bruce MacDonald
536028c35a
better error message when model not found on pull
2023-07-24 17:48:17 -04:00
Bruce MacDonald
abf614804b
remove file on digest mismatch
2023-07-24 21:59:12 +02:00
Bruce MacDonald
a0dbbb23c4
truncate file size on resume
2023-07-24 21:58:32 +02:00
Bruce MacDonald
0fd6278446
do not panic server if file cannot be opened
2023-07-24 15:24:34 -04:00
Bruce MacDonald
abfc73d31e
make response errors unique for error trace
2023-07-24 15:04:21 -04:00
Bruce MacDonald
5a5ca8e7ff
remove file on digest mismatch
2023-07-24 14:53:01 -04:00
Bruce MacDonald
fdbef6c95e
truncate file size on resume
2023-07-24 14:36:19 -04:00
Patrick Devine
4cb42ca55e
add copy command ( #191 )
2023-07-24 11:27:28 -04:00
Patrick Devine
88c55199f8
change push to chunked uploads from monolithic ( #179 )
2023-07-22 17:31:26 -07:00
Michael Yang
8609db77ea
use gin-contrib/cors middleware
2023-07-22 09:39:08 -07:00
Patrick Devine
6d6b0d3321
change error handler behavior and fix error when a model isn't found ( #173 )
2023-07-21 23:02:12 -07:00
Michael Yang
20a5d99f77
fix vars.First
2023-07-21 20:45:32 -07:00
Patrick Devine
3b43cc019a
fix extended tag names ( #171 )
2023-07-21 20:27:25 -07:00
Patrick Devine
b8421dce3d
get the proper path for blobs to delete ( #168 )
2023-07-21 17:30:40 -07:00
Patrick Devine
9f6e97865c
allow pushing/pulling to insecure registries ( #157 )
2023-07-21 15:42:19 -07:00
Bruce MacDonald
7ba1308595
Merge pull request #147 from jmorganca/brucemacd/cli-err-display
...
Improve CLI error display
2023-07-21 16:10:19 +02:00
Patrick Devine
e7a393de54
add rm command for models ( #151 )
2023-07-20 16:09:23 -07:00
Michael Yang
6cea2061ec
windows: fix model pulling
2023-07-20 12:35:04 -07:00
Michael Yang
2832801c2a
Merge pull request #91 from jmorganca/fix-stream-errors
...
fix stream errors
2023-07-20 12:21:59 -07:00
Michael Yang
992892866b
Merge pull request #145 from jmorganca/verify-digest
...
verify blob digest
2023-07-20 12:14:21 -07:00
Michael Yang
1f27d7f1b8
fix stream errors
2023-07-20 12:12:08 -07:00
Michael Yang
bf198c3918
verify blob digest
2023-07-20 11:53:57 -07:00
Bruce MacDonald
09dc6273e3
suppress error when running list before pulling image
2023-07-20 20:53:09 +02:00
Bruce MacDonald
3ec4ebc562
remove unused code
2023-07-20 20:18:00 +02:00
Jeffrey Morgan
d59b164fa2
add prompt back to parser
2023-07-20 01:13:30 -07:00
Jeffrey Morgan
75e508e1d6
remove old templates
2023-07-19 23:47:13 -07:00
Michael Yang
6f046dbf18
Update images.go ( #134 )
2023-07-19 23:46:01 -07:00
Michael Yang
60b4db6389
add .First
2023-07-19 23:24:32 -07:00
Michael Yang
ca210ba480
handle vnd.ollama.image.prompt for compat
2023-07-19 23:24:32 -07:00
Michael Yang
df146c41e2
separate prompt into template and system
2023-07-19 23:24:31 -07:00
Jeffrey Morgan
2d305fa99a
allow relative paths in FROM
instruction
2023-07-19 21:55:15 -07:00
Jeffrey Morgan
4ca7c4be1f
dont consume reader when calculating digest
2023-07-19 00:47:55 -07:00
Patrick Devine
572fc9099f
add license layers to the parser ( #116 )
2023-07-18 22:49:38 -07:00
Michael Yang
68df36ae50
fix pull 0 bytes on completed layer
2023-07-18 19:38:11 -07:00
Michael Yang
6e36f948df
Merge pull request #109 from jmorganca/fix-create-memory
...
fix memory leak in create
2023-07-18 17:25:19 -07:00
Michael Yang
553fa39fe8
fix memory leak in create
2023-07-18 17:14:17 -07:00
Patrick Devine
9e15635c2d
attempt two for skipping files in the file walk ( #105 )
2023-07-18 15:37:01 -07:00
Patrick Devine
9658a5043b
skip files in the list if we can't get the correct model path ( #100 )
2023-07-18 12:39:08 -07:00
Michael Yang
f20426a768
fix mkdir blob path
2023-07-18 11:24:19 -07:00
Patrick Devine
5bea29f610
add new list command ( #97 )
2023-07-18 09:09:45 -07:00
Patrick Devine
4a28a2f093
add modelpaths ( #96 )
2023-07-17 22:44:21 -07:00
Michael Yang
c7dd52271c
remove debugging messages
2023-07-17 14:17:34 -07:00
Michael Yang
53d0052c6c
unavoid unnecessary type conversion
2023-07-17 12:35:03 -07:00
Michael Yang
28a136e9a3
modelfile params
2023-07-17 12:35:03 -07:00
Michael Yang
3862a51a6a
create directories if they do not exist
2023-07-17 11:18:48 -07:00
Michael Yang
bcb612a30a
fix file paths for windows
2023-07-17 10:47:47 -07:00
Patrick Devine
2fb52261ad
basic distribution w/ push/pull ( #78 )
...
* basic distribution w/ push/pull
* add the parser
* add create, pull, and push
* changes to the parser, FROM line, and fix commands
* mkdirp new manifest directories
* make `blobs` directory if it does not exist
* fix go warnings
* add progressbar for model pulls
* move model struct
---------
Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>
2023-07-16 17:02:22 -07:00
Michael Yang
743e957d88
use filepath for os compat
2023-07-14 17:27:14 -07:00
Michael Yang
5ade3db040
fix race
...
block on write which only returns when the channel is closed. this is
contrary to the previous arrangement where the handler may return but
the stream hasn't finished writing. it can lead to the client receiving
unexpected responses (since the request has been handled) or worst case
a nil-pointer dereference as the stream tries to flush a nil writer
2023-07-14 15:10:46 -07:00
Michael Yang
1775647f76
continue conversation
...
feed responses back into the llm
2023-07-13 17:13:00 -07:00
Michael Yang
05e08d2310
return more info in generate response
2023-07-13 09:37:32 -07:00
Michael Yang
31590284a7
fix route
2023-07-12 19:21:49 -07:00
Michael Yang
2666d3c206
fix pull race
2023-07-12 19:07:23 -07:00
Michael Yang
0944b01e7d
pull fixes
2023-07-12 09:55:07 -07:00
Michael Yang
a806b03f62
no errgroup
2023-07-11 14:58:10 -07:00
Michael Yang
948323fa78
rename partial file
2023-07-11 13:50:26 -07:00
Michael Yang
e243329e2e
check api status
2023-07-11 13:42:05 -07:00
Michael Yang
2a66a1164a
common stream producer
2023-07-11 13:42:05 -07:00
Michael Yang
fd4792ec56
call llama.cpp directly from go
2023-07-11 11:59:18 -07:00
Jeffrey Morgan
a3ec1ec2a0
consistent error handling for pull and generate
2023-07-10 21:34:15 -07:00
Michael Yang
edba935d67
return error in generate response
2023-07-10 13:30:10 -07:00
Bruce MacDonald
f5e2e150b8
allow overriding default generate options
2023-07-10 20:58:02 +02:00
Jeffrey Morgan
74e92d1258
add basic /
route for server
2023-07-07 23:46:15 -04:00
Bruce MacDonald
f533f85d44
pr feedback
...
- move error check to api client pull
- simplify error check in generate
- return nil on any pull error
2023-07-07 17:12:02 -04:00
Bruce MacDonald
61dd87bd90
if directory cannot be resolved, do not fail
2023-07-07 15:27:43 -04:00
Bruce MacDonald
b24be8c6b3
update directory url
2023-07-07 15:13:41 -04:00
Michael Yang
053739d19f
no prompt on empty line
2023-07-07 11:01:44 -07:00
Patrick Devine
3f1b7177f2
pass model and predict options
2023-07-07 09:34:05 -07:00
Michael Yang
b0618a466e
generate progress
2023-07-06 17:07:40 -07:00
Michael Yang
c4b9e84945
progress
2023-07-06 17:07:40 -07:00
Michael Yang
15c114decb
fix prompt templates
2023-07-06 17:03:18 -07:00
Michael Yang
0637632258
simple pull response
2023-07-06 16:34:44 -04:00
Michael Yang
dd960d1d5e
update generate response
2023-07-06 16:34:44 -04:00
Bruce MacDonald
d436d51c78
clean up model pull
2023-07-06 16:34:44 -04:00
Bruce MacDonald
c9f45abef3
resumable downloads
2023-07-06 16:34:44 -04:00
Michael Yang
9b8a456c7d
embed templates
2023-07-06 16:34:44 -04:00
Bruce MacDonald
7cf5905063
display pull progress
2023-07-06 16:34:44 -04:00
Michael Yang
580fe8951c
free llama model
2023-07-06 16:34:44 -04:00
Michael Yang
68e6b4550c
use prompt templates
2023-07-06 16:34:44 -04:00
Bruce MacDonald
a6494f8211
pull models
2023-07-06 16:34:44 -04:00
Michael Yang
1b7183c5a1
enable metal gpu acceleration
...
ggml-metal.metal must be in the same directory as the ollama binary
otherwise llama.cpp will not be able to find it and load it.
1. go generate llama/llama_metal.go
2. go build .
3. ./ollama serve
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
0998d4f0a4
remove debug print statements
2023-07-06 16:34:44 -04:00
Bruce MacDonald
8ea5e5e147
separate routes
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
fd962a36e5
client updates
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
9164981d72
move prompt templates out of python bindings
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
6093a88c1a
add llama.cpp go bindings
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
76cb60d496
wip go engine
...
Co-authored-by: Patrick Devine <pdevine@sonic.net>
2023-07-06 16:34:44 -04:00
Jeffrey Morgan
b361fa72ec
reorganize directories
2023-06-25 13:08:03 -04:00
Jeffrey Morgan
d3709f85b5
build server into desktop app
2023-06-25 00:30:02 -04:00
Jeffrey Morgan
369108e1ad
Add dependencies to requirements.txt
2023-06-24 21:13:26 -04:00
Michael Chiang
5142ba2dad
Update README.md
2023-06-23 22:51:54 -04:00
Bruce MacDonald
c5bafaff54
package server with client
2023-06-23 18:38:22 -04:00
Bruce MacDonald
f0eee3faa0
build server executable
2023-06-23 17:23:30 -04:00
Bruce MacDonald
ebec1c61db
load and unload model endpoints
2023-06-23 14:47:57 -04:00
Bruce MacDonald
0758cb2d4b
llama server wrapper
2023-06-23 13:10:13 -04:00
Jeffrey Morgan
8fa91332fa
initial commit
2023-06-22 18:31:40 -04:00