From b662d9fd8cc858ab359577d4df773ce02011a8b3 Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Fri, 4 Aug 2023 11:55:00 -0700 Subject: [PATCH 01/16] starting to build out some docs Signed-off-by: Matt Williams --- docs/README.md | 6 ++++++ docs/api.md | 21 +++++++++++++++++++++ 2 files changed, 27 insertions(+) create mode 100644 docs/README.md create mode 100644 docs/api.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..ea11fe3f --- /dev/null +++ b/docs/README.md @@ -0,0 +1,6 @@ +# Documentation + +- [Modelfile](./modelfile.md) +- [How we store Models](./modelstorage.md) +- [How to develop Ollama](./development.md) +- [API](./api.md) diff --git a/docs/api.md b/docs/api.md new file mode 100644 index 00000000..c2b9f8ca --- /dev/null +++ b/docs/api.md @@ -0,0 +1,21 @@ +[Documentation Home](./README.md) + +# API + +## Generate a Prompt +**/api/generate** + +## Create a Model +**/api/create** + +## List Local Models +**/api/list** + +## Copy a Model +**/api/copy** + +## Pull a Model +**/api/pull** + +## Push a Model +**/api/push** \ No newline at end of file From b31104768c7cf8d9c40be2898f817c6c622bab20 Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Fri, 4 Aug 2023 12:27:47 -0700 Subject: [PATCH 02/16] filling out generate Signed-off-by: Matt Williams --- docs/api.md | 58 +++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 54 insertions(+), 4 deletions(-) diff --git a/docs/api.md b/docs/api.md index c2b9f8ca..cdde3d52 100644 --- a/docs/api.md +++ b/docs/api.md @@ -3,19 +3,69 @@ # API ## Generate a Prompt -**/api/generate** +**POST /api/generate** +The **Generate** endpoint takes a JSON object with the following fields: + +``` +{ + Model: "modelname", + Prompt: "prompt", + Context: "context", +} +``` + +Context is optional, but is used to provide additional context, such as memory of earlier prompts. + +The response is a stream of JSON objects with the following fields: + +``` +{ + "model": "modelname", + "created_at": "2023-08-04T08:52:19.385406455-07:00" + "response": "the current token", + "done": false +} +``` + +| field | description | +| --- | --- | +| model | the name of the model | +| created_at | the time the response was generated | +| response | the current token | +| done | whether the response is complete | ## Create a Model -**/api/create** +**POST /api/create** ## List Local Models -**/api/list** +**GET /api/tags** + +### Return Object +``` +{ + "models": [ + { + "name": "modelname:tags", + "modified_at": "2023-08-04T08:52:19.385406455-07:00", + "size": size + } + ] + +} +``` ## Copy a Model **/api/copy** +## Delete a Model +**/api/delete** + ## Pull a Model **/api/pull** ## Push a Model -**/api/push** \ No newline at end of file +**/api/push** + +## Heartbeat +**/** + From 31edc829fc4c3535e3c6019ba0bb996cc7264854 Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Fri, 4 Aug 2023 12:30:23 -0700 Subject: [PATCH 03/16] continuing Signed-off-by: Matt Williams --- docs/api.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/api.md b/docs/api.md index cdde3d52..93a50700 100644 --- a/docs/api.md +++ b/docs/api.md @@ -5,6 +5,8 @@ ## Generate a Prompt **POST /api/generate** +**Generate** is the main endpoint that you will use when working with Ollama. This is used to generate a response to a prompt sent to a model. + The **Generate** endpoint takes a JSON object with the following fields: ``` From d93e2f9210831f93ff2a9839e434285cd3455385 Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Fri, 4 Aug 2023 12:38:58 -0700 Subject: [PATCH 04/16] fleshing out response Signed-off-by: Matt Williams --- docs/api.md | 81 ++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 74 insertions(+), 7 deletions(-) diff --git a/docs/api.md b/docs/api.md index 93a50700..c8eb734a 100644 --- a/docs/api.md +++ b/docs/api.md @@ -3,6 +3,7 @@ # API ## Generate a Prompt + **POST /api/generate** **Generate** is the main endpoint that you will use when working with Ollama. This is used to generate a response to a prompt sent to a model. @@ -17,7 +18,7 @@ The **Generate** endpoint takes a JSON object with the following fields: } ``` -Context is optional, but is used to provide additional context, such as memory of earlier prompts. +Context is optional, but is used to provide additional context, such as memory of earlier prompts. The response is a stream of JSON objects with the following fields: @@ -30,19 +31,81 @@ The response is a stream of JSON objects with the following fields: } ``` -| field | description | -| --- | --- | -| model | the name of the model | +The final response in the stream also includes the context and what is usually seen in the output from verbose mode. For example: + +``` +{ + "model":"orca", + "created_at":"2023-08-04T19:22:45.499127Z", + "done":true, + "context":[1,31822,1,13,8458,31922 ... 382,871,550,389,266,7661,31844,382,820,541,4842,1954,661,645,590,3465,31843,2], + "total_duration":5589157167, + "load_duration":3013701500, + "sample_count":114, + "sample_duration":81442000, + "prompt_eval_count":46, + "prompt_eval_duration":1160282000, + "eval_count":113, + "eval_duration":1325948000 +} +``` + +| field | description | +| ---------- | ----------------------------------- | +| model | the name of the model | | created_at | the time the response was generated | -| response | the current token | -| done | whether the response is complete | +| response | the current token | +| done | whether the response is complete | +| context | vectorize context that can be supplied in the next request to continue the conversation | +| total_duration | total time spent generating the response | +| load_duration | time spent loading the model | +| sample_count | number of samples generated | +| sample_duration | time spent generating samples | +| prompt_eval_count | number of times the prompt was evaluated | +| prompt_eval_duration | time spent evaluating the prompt | +| eval_count | number of times the response was evaluated | +| eval_duration | time spent evaluating the response | + + + +### Example Request + +```curl +curl --location --request POST 'http://localhost:11434/api/generate' \ +--header 'Content-Type: text/plain' \ +--data-raw '{ + "model": "orca", + "prompt": "why is the sky blue" +}' +``` + +### Example Response + +```json +{"model":"orca","created_at":"2023-08-04T19:22:44.085127Z","response":" The","done":false} +{"model":"orca","created_at":"2023-08-04T19:22:44.176425Z","response":" sky","done":false} +{"model":"orca","created_at":"2023-08-04T19:22:44.18883Z","response":" appears","done":false} +{"model":"orca","created_at":"2023-08-04T19:22:44.200852Z","response":" blue","done":false} +{"model":"orca","created_at":"2023-08-04T19:22:44.213644Z","response":" because","done":false} +{"model":"orca","created_at":"2023-08-04T19:22:44.225706Z","response":" of","done":false} +{"model":"orca","created_at":"2023-08-04T19:22:44.237686Z","response":" a","done":false} +. +. +. +{"model":"orca","created_at":"2023-08-04T19:22:45.487113Z","response":".","done":false} +{"model":"orca","created_at":"2023-08-04T19:22:45.499127Z","done":true,"context":[1,31822,1,13,8458,31922,3244,31871,13,3838,397,363,7421,8825,342,5243,10389,5164,828,31843,9530,362,988,362,365,473,31843,13,13,8458,31922,9779,31871,13,23712,322,266,7661,4842,13,13,8458,31922,13166,31871,13,347,7661,4725,4842,906,287,260,12329,1676,6697,27554,27289,31843,4025,2990,322,985,550,287,260,9949,287,8286,31844,10990,427,2729,289,399,20036,31843,1408,21062,16858,266,4556,31876,31829,7965,31844,357,19322,16450,287,1900,859,362,22329,291,11944,31843,1872,16450,397,988,5497,661,266,23893,287,266,1954,31844,560,526,640,3304,266,1954,288,484,11468,31843,1813,31844,4842,1954,470,260,13830,23893,661,590,8286,31844,560,357,322,18752,541,4083,31843,672,1901,342,662,382,871,550,389,266,7661,31844,382,820,541,4842,1954,661,645,590,3465,31843,2],"total_duration":5589157167,"load_duration":3013701500,"sample_count":114,"sample_duration":81442000,"prompt_eval_count":46,"prompt_eval_duration":1160282000,"eval_count":113,"eval_duration":1325948000} +``` + ## Create a Model + **POST /api/create** ## List Local Models + **GET /api/tags** ### Return Object + ``` { "models": [ @@ -57,17 +120,21 @@ The response is a stream of JSON objects with the following fields: ``` ## Copy a Model + **/api/copy** ## Delete a Model + **/api/delete** ## Pull a Model + **/api/pull** ## Push a Model + **/api/push** ## Heartbeat -**/** +**/** From a101fe51a74a88f310dd966fa910bbcec51771d8 Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Fri, 4 Aug 2023 12:56:41 -0700 Subject: [PATCH 05/16] clean up Signed-off-by: Matt Williams --- docs/api.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/docs/api.md b/docs/api.md index c8eb734a..8aa7f926 100644 --- a/docs/api.md +++ b/docs/api.md @@ -14,12 +14,9 @@ The **Generate** endpoint takes a JSON object with the following fields: { Model: "modelname", Prompt: "prompt", - Context: "context", } ``` -Context is optional, but is used to provide additional context, such as memory of earlier prompts. - The response is a stream of JSON objects with the following fields: ``` @@ -38,7 +35,6 @@ The final response in the stream also includes the context and what is usually s "model":"orca", "created_at":"2023-08-04T19:22:45.499127Z", "done":true, - "context":[1,31822,1,13,8458,31922 ... 382,871,550,389,266,7661,31844,382,820,541,4842,1954,661,645,590,3465,31843,2], "total_duration":5589157167, "load_duration":3013701500, "sample_count":114, @@ -56,7 +52,6 @@ The final response in the stream also includes the context and what is usually s | created_at | the time the response was generated | | response | the current token | | done | whether the response is complete | -| context | vectorize context that can be supplied in the next request to continue the conversation | | total_duration | total time spent generating the response | | load_duration | time spent loading the model | | sample_count | number of samples generated | @@ -93,7 +88,7 @@ curl --location --request POST 'http://localhost:11434/api/generate' \ . . {"model":"orca","created_at":"2023-08-04T19:22:45.487113Z","response":".","done":false} -{"model":"orca","created_at":"2023-08-04T19:22:45.499127Z","done":true,"context":[1,31822,1,13,8458,31922,3244,31871,13,3838,397,363,7421,8825,342,5243,10389,5164,828,31843,9530,362,988,362,365,473,31843,13,13,8458,31922,9779,31871,13,23712,322,266,7661,4842,13,13,8458,31922,13166,31871,13,347,7661,4725,4842,906,287,260,12329,1676,6697,27554,27289,31843,4025,2990,322,985,550,287,260,9949,287,8286,31844,10990,427,2729,289,399,20036,31843,1408,21062,16858,266,4556,31876,31829,7965,31844,357,19322,16450,287,1900,859,362,22329,291,11944,31843,1872,16450,397,988,5497,661,266,23893,287,266,1954,31844,560,526,640,3304,266,1954,288,484,11468,31843,1813,31844,4842,1954,470,260,13830,23893,661,590,8286,31844,560,357,322,18752,541,4083,31843,672,1901,342,662,382,871,550,389,266,7661,31844,382,820,541,4842,1954,661,645,590,3465,31843,2],"total_duration":5589157167,"load_duration":3013701500,"sample_count":114,"sample_duration":81442000,"prompt_eval_count":46,"prompt_eval_duration":1160282000,"eval_count":113,"eval_duration":1325948000} +{"model":"orca","created_at":"2023-08-04T19:22:45.499127Z","done":true,"total_duration":5589157167,"load_duration":3013701500,"sample_count":114,"sample_duration":81442000,"prompt_eval_count":46,"prompt_eval_duration":1160282000,"eval_count":113,"eval_duration":1325948000} ``` ## Create a Model From c18be72a3ba6a72d9ccd4fbec8addd198d8c52e9 Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Fri, 4 Aug 2023 16:08:11 -0700 Subject: [PATCH 06/16] complete 1st draft of api docs Signed-off-by: Matt Williams --- docs/api.md | 297 ++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 264 insertions(+), 33 deletions(-) diff --git a/docs/api.md b/docs/api.md index 8aa7f926..5f7cdefb 100644 --- a/docs/api.md +++ b/docs/api.md @@ -2,24 +2,37 @@ # API +- [Generate a Prompt](#generate-a-prompt) +- [Create a Model](#create-a-model) +- [List Local Models](#list-local-models) +- [Copy a Model](#copy-a-model) +- [Delete a Model](#delete-a-model) +- [Pull a Model](#pull-a-model) + ## Generate a Prompt **POST /api/generate** +### Description + **Generate** is the main endpoint that you will use when working with Ollama. This is used to generate a response to a prompt sent to a model. +### Request + The **Generate** endpoint takes a JSON object with the following fields: -``` +```JSON { Model: "modelname", Prompt: "prompt", } ``` +### Response + The response is a stream of JSON objects with the following fields: -``` +```JSON { "model": "modelname", "created_at": "2023-08-04T08:52:19.385406455-07:00" @@ -30,7 +43,7 @@ The response is a stream of JSON objects with the following fields: The final response in the stream also includes the context and what is usually seen in the output from verbose mode. For example: -``` +```JSON { "model":"orca", "created_at":"2023-08-04T19:22:45.499127Z", @@ -46,26 +59,26 @@ The final response in the stream also includes the context and what is usually s } ``` -| field | description | -| ---------- | ----------------------------------- | -| model | the name of the model | -| created_at | the time the response was generated | -| response | the current token | -| done | whether the response is complete | -| total_duration | total time spent generating the response | -| load_duration | time spent loading the model | -| sample_count | number of samples generated | -| sample_duration | time spent generating samples | -| prompt_eval_count | number of times the prompt was evaluated | -| prompt_eval_duration | time spent evaluating the prompt | -| eval_count | number of times the response was evaluated | -| eval_duration | time spent evaluating the response | +| field | description | +| -------------------- | ------------------------------------------ | +| model | the name of the model | +| created_at | the time the response was generated | +| response | the current token | +| done | whether the response is complete | +| total_duration | total time spent generating the response | +| load_duration | time spent loading the model | +| sample_count | number of samples generated | +| sample_duration | time spent generating samples | +| prompt_eval_count | number of times the prompt was evaluated | +| prompt_eval_duration | time spent evaluating the prompt | +| eval_count | number of times the response was evaluated | +| eval_duration | time spent evaluating the response | +### Example +#### Request -### Example Request - -```curl +```shell curl --location --request POST 'http://localhost:11434/api/generate' \ --header 'Content-Type: text/plain' \ --data-raw '{ @@ -74,7 +87,7 @@ curl --location --request POST 'http://localhost:11434/api/generate' \ }' ``` -### Example Response +#### Response ```json {"model":"orca","created_at":"2023-08-04T19:22:44.085127Z","response":" The","done":false} @@ -84,9 +97,7 @@ curl --location --request POST 'http://localhost:11434/api/generate' \ {"model":"orca","created_at":"2023-08-04T19:22:44.213644Z","response":" because","done":false} {"model":"orca","created_at":"2023-08-04T19:22:44.225706Z","response":" of","done":false} {"model":"orca","created_at":"2023-08-04T19:22:44.237686Z","response":" a","done":false} -. -. -. +... {"model":"orca","created_at":"2023-08-04T19:22:45.487113Z","response":".","done":false} {"model":"orca","created_at":"2023-08-04T19:22:45.499127Z","done":true,"total_duration":5589157167,"load_duration":3013701500,"sample_count":114,"sample_duration":81442000,"prompt_eval_count":46,"prompt_eval_duration":1160282000,"eval_count":113,"eval_duration":1325948000} ``` @@ -95,13 +106,76 @@ curl --location --request POST 'http://localhost:11434/api/generate' \ **POST /api/create** +### Description + +**Create** takes a path to a Modelfile and creates a model. The Modelfile is documented [here](./modelfile.md). + +### Request + +The **Create** endpoint takes a JSON object with the following fields: + +```JSON +{ + "name": "modelname", + "path": "path to Modelfile" +} +``` + +### Response + +The response is a stream of JSON objects that have a single key/value pair for status. For example: + +```JSON +{ + "status": "parsing modelfile" +} +``` + +### Example + +#### Request + +```shell +curl --location --request POST 'http://localhost:11434/api/create' \ +--header 'Content-Type: text/plain' \ +--data-raw '{ + "name": "myCoolModel", + "path": "/Users/matt/ollamamodelfiles/sentiments" +}' +``` + +#### Response + +```JSON +{"status":"parsing modelfile"} +{"status":"looking for model"} +{"status":"creating model template layer"} +{"status":"creating config layer"} +{"status":"using already created layer sha256:e84705205f71dd55be7b24a778f248f0eda9999a125d313358c087e092d83148"} +{"status":"using already created layer sha256:93ca9b3d83dc541f11062c0b994ae66a7b327146f59a9564aafef4a4c15d1ef5"} +{"status":"writing layer sha256:d3fe6fb39620a477da7720c5fa00abe269a018a9675a726320e18122b7142ee7"} +{"status":"writing layer sha256:16cc83359b0395026878b41662f7caef433f5260b5d49a3257312b6417b7d8a8"} +{"status":"writing manifest"} +{"status":"success"} +``` + ## List Local Models **GET /api/tags** -### Return Object +### Description -``` +**List** will list out all the models on the that have been created or pulled locally. + +### Request + +The **List** endpoint takes no parameters and is a simple GET request. + +### Response + +The response is a JSON object with a single key/value pair for models. For example: + +```JSON { "models": [ { @@ -110,26 +184,183 @@ curl --location --request POST 'http://localhost:11434/api/generate' \ "size": size } ] +} +``` +### Example + +#### Request + +```shell +curl --location --request GET 'http://localhost:11434/api/tags' +``` + +#### Response + +```JSON +{ + "models": [ + { + "name": "llama2:70b", + "modified_at": "2023-08-04T08:52:19.385406455-07:00", + "size": 38871966966 + }, + { + "name": "llama2:70b-chat-q4_0", + "modified_at": "2023-08-04T09:21:27.703371485-07:00", + "size": 38871974480 + }, + { + "name": "midjourney-prompter:latest", + "modified_at": "2023-08-04T08:45:46.399609053-07:00", + "size": 7323311708 + }, + { + "name": "raycast_orca:3b", + "modified_at": "2023-08-04T06:23:20.10832636-07:00", + "size": 1928446602 + }, + { + "name": "stablebeluga:13b-q4_K_M", + "modified_at": "2023-08-04T09:48:26.416547463-07:00", + "size": 7865679045 + } + ] } ``` ## Copy a Model -**/api/copy** +**POST /api/copy** + +### Description + +**Copy** will copy a model from one name to another. This is useful for creating a new model from an existing model. It is often used as the first step to renaming a model. + +### Request + +The **Copy** endpoint takes a JSON object with the following fields: + +```JSON +{ + "from": "modelname", + "to": "newmodelname" +} +``` + +### Response + +There is no response other than a 200 status code. + +### Example + +#### Request + +```shell +curl --location --request POST 'http://localhost:11434/api/copy' \ +--header 'Content-Type: text/plain' \ +--data-raw '{ + "source": "MyCoolModel", + "destination": "ADifferentModel" +}' +``` + +#### Response + +No response is returned other than a 200 status code. ## Delete a Model -**/api/delete** +**DEL /api/delete** + +### Description + +**Delete** will delete a model from the local machine. This is useful for cleaning up models that are no longer needed. + +### Request + +The **Delete** endpoint takes a JSON object with a single key/value pair for modelname. For example: + +```JSON +{ + "model": "modelname" +} +``` + +### Response + +No response is returned other than a 200 status code. + +### Example + +#### Request + +```shell +curl --location --request DELETE 'http://localhost:11434/api/delete' \ +--header 'Content-Type: text/plain' \ +--data-raw '{ + "name": "adifferentModel" +}' +``` + +#### Response + +No response is returned other than a 200 status code. ## Pull a Model -**/api/pull** +**POST /api/pull** -## Push a Model +### Description -**/api/push** +**Pull** will pull a model from a remote registry. This is useful for getting a model from the Ollama registry and in the future from alternate registries. -## Heartbeat +### Request -**/** +The **Pull** endpoint takes a JSON object with the following fields: + +```JSON +{ + "name": "modelname", + "registry": "registryname" +} +``` + +### Response + +The response is a stream of JSON objects with the following format: + +```JSON +{ + "status":"downloading digestname", + "digest":"digestname", + "total":2142590208 +} +``` + +### Example + +#### Request + +```shell +curl --location --request POST 'http://localhost:11434/api/pull' \ +--header 'Content-Type: text/plain' \ +--data-raw '{ + "name": "orca:3b-q4_1" +}' +``` + +#### Response + +```JSON +{"status":"pulling manifest"} +{"status":"downloading sha256:63151c63f792939bb4a40b35f37ea06e047c02486399d1742113aaefd0d33e29","digest":"sha256:63151c63f792939bb4a40b35f37ea06e047c02486399d1742113aaefd0d33e29","total":2142590208} +{"status":"downloading sha256:63151c63f792939bb4a40b35f37ea06e047c02486399d1742113aaefd0d33e29","digest":"sha256:63151c63f792939bb4a40b35f37ea06e047c02486399d1742113aaefd0d33e29","total":2142590208,"completed":1048576} +... +{"status":"downloading sha256:20714f2ebe4be44313358bfa58556d783652398ed47f12178914c706c4ad12c4","digest":"sha256:20714f2ebe4be44313358bfa58556d783652398ed47f12178914c706c4ad12c4","total":299} +{"status":"downloading sha256:20714f2ebe4be44313358bfa58556d783652398ed47f12178914c706c4ad12c4","digest":"sha256:20714f2ebe4be44313358bfa58556d783652398ed47f12178914c706c4ad12c4","total":299,"completed":299} +{"status":"verifying sha256 digest"} +{"status":"writing manifest"} +{"status":"success"} + +``` From 38a9b1618c1d5843c936af2e8645300cee4a4656 Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Fri, 4 Aug 2023 16:09:07 -0700 Subject: [PATCH 07/16] missed some quotes Signed-off-by: Matt Williams --- docs/api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/api.md b/docs/api.md index 5f7cdefb..6a4f8f32 100644 --- a/docs/api.md +++ b/docs/api.md @@ -23,8 +23,8 @@ The **Generate** endpoint takes a JSON object with the following fields: ```JSON { - Model: "modelname", - Prompt: "prompt", + "model": "modelname", + "prompt": "prompt", } ``` From 5c82bf95d1c9424470db62a917225920a423af6e Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Fri, 4 Aug 2023 17:12:24 -0700 Subject: [PATCH 08/16] Update docs/api.md Co-authored-by: Michael Yang --- docs/api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/api.md b/docs/api.md index 6a4f8f32..05efa5a6 100644 --- a/docs/api.md +++ b/docs/api.md @@ -165,7 +165,7 @@ curl --location --request POST 'http://localhost:11434/api/create' \ ### Description -**List** will list out all the models on the that have been created or pulled locally. +**List** will list out the models that are available locally. ### Request From 123fdeb9190e999a397525aea969ee943a4c0392 Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Fri, 4 Aug 2023 17:38:52 -0700 Subject: [PATCH 09/16] Update docs/api.md Co-authored-by: Michael Yang --- docs/api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/api.md b/docs/api.md index 05efa5a6..d966b4d5 100644 --- a/docs/api.md +++ b/docs/api.md @@ -243,8 +243,8 @@ The **Copy** endpoint takes a JSON object with the following fields: ```JSON { - "from": "modelname", - "to": "newmodelname" + "source": "modelname", + "destination": "newmodelname" } ``` From ac1b04f271fae08b12fcea74fb92a019ba499b77 Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Fri, 4 Aug 2023 17:40:52 -0700 Subject: [PATCH 10/16] Update docs/api.md Co-authored-by: Michael Yang --- docs/api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/api.md b/docs/api.md index d966b4d5..283ddac9 100644 --- a/docs/api.md +++ b/docs/api.md @@ -271,7 +271,7 @@ No response is returned other than a 200 status code. ## Delete a Model -**DEL /api/delete** +**DELETE /api/delete** ### Description From 2544b8afa1e8237a436eb5559736a9aafe8d166b Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Fri, 4 Aug 2023 17:41:28 -0700 Subject: [PATCH 11/16] update as per Mike's comments Signed-off-by: Matt Williams --- docs/README.md | 1 - docs/api.md | 40 ++++++++++++++++++++++------------------ 2 files changed, 22 insertions(+), 19 deletions(-) diff --git a/docs/README.md b/docs/README.md index ea11fe3f..84f7dd8a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,6 +1,5 @@ # Documentation - [Modelfile](./modelfile.md) -- [How we store Models](./modelstorage.md) - [How to develop Ollama](./development.md) - [API](./api.md) diff --git a/docs/api.md b/docs/api.md index 283ddac9..4f79dc62 100644 --- a/docs/api.md +++ b/docs/api.md @@ -24,10 +24,15 @@ The **Generate** endpoint takes a JSON object with the following fields: ```JSON { "model": "modelname", - "prompt": "prompt", + "prompt": "You are a software engineer working on building docs for Ollama.", + "options": { + "temperature": 0.7, + } } ``` +**Options** can include any of the parameters listed in the [Modelfile](./modelfile.mdvalid-parameters-and-values) documentation. The only required parameter is **model**. If no **prompt** is provided, the model will generate a response to an empty prompt. If no **options** are provided, the model will use the default options from the Modelfile of the parent model. + ### Response The response is a stream of JSON objects with the following fields: @@ -59,20 +64,20 @@ The final response in the stream also includes the context and what is usually s } ``` -| field | description | -| -------------------- | ------------------------------------------ | -| model | the name of the model | -| created_at | the time the response was generated | -| response | the current token | -| done | whether the response is complete | -| total_duration | total time spent generating the response | -| load_duration | time spent loading the model | -| sample_count | number of samples generated | -| sample_duration | time spent generating samples | -| prompt_eval_count | number of times the prompt was evaluated | -| prompt_eval_duration | time spent evaluating the prompt | -| eval_count | number of times the response was evaluated | -| eval_duration | time spent evaluating the response | +| field | description | +| -------------------- | ------------------------------------------------------- | +| model | the name of the model | +| created_at | the time the response was generated | +| response | the current token | +| done | whether the response is complete | +| total_duration | total time in nanoseconds spent generating the response | +| load_duration | time spent in nanoseconds loading the model | +| sample_count | number of samples generated | +| sample_duration | time spent generating samples | +| prompt_eval_count | number of times the prompt was evaluated | +| prompt_eval_duration | time spent in nanoseconds evaluating the prompt | +| eval_count | number of times the response was evaluated | +| eval_duration | time in nanoseconds spent evaluating the response | ### Example @@ -117,7 +122,7 @@ The **Create** endpoint takes a JSON object with the following fields: ```JSON { "name": "modelname", - "path": "path to Modelfile" + "path": "absolute path to Modelfile" } ``` @@ -321,8 +326,7 @@ The **Pull** endpoint takes a JSON object with the following fields: ```JSON { - "name": "modelname", - "registry": "registryname" + "name": "modelname" } ``` From 8a45359ec6493bcc3fcb9db3bb40aa8a4b36dc0c Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Mon, 7 Aug 2023 07:33:05 -0700 Subject: [PATCH 12/16] Update docs/api.md Co-authored-by: Jeffrey Morgan --- docs/api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/api.md b/docs/api.md index 4f79dc62..6abb01ff 100644 --- a/docs/api.md +++ b/docs/api.md @@ -186,7 +186,7 @@ The response is a JSON object with a single key/value pair for models. For examp { "name": "modelname:tags", "modified_at": "2023-08-04T08:52:19.385406455-07:00", - "size": size + "size": 7323310500 } ] } From 4904cd8bcd2c912228fc69650cb67de4b56ce1da Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Mon, 7 Aug 2023 07:40:38 -0700 Subject: [PATCH 13/16] update simpler code samples Signed-off-by: Matt Williams --- docs/api.md | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/docs/api.md b/docs/api.md index 6abb01ff..d0e0da9c 100644 --- a/docs/api.md +++ b/docs/api.md @@ -84,9 +84,8 @@ The final response in the stream also includes the context and what is usually s #### Request ```shell -curl --location --request POST 'http://localhost:11434/api/generate' \ ---header 'Content-Type: text/plain' \ ---data-raw '{ +curl -X POST 'http://localhost:11434/api/generate' -d \ +'{ "model": "orca", "prompt": "why is the sky blue" }' @@ -197,7 +196,7 @@ The response is a JSON object with a single key/value pair for models. For examp #### Request ```shell -curl --location --request GET 'http://localhost:11434/api/tags' +curl 'http://localhost:11434/api/tags' ``` #### Response @@ -262,9 +261,8 @@ There is no response other than a 200 status code. #### Request ```shell -curl --location --request POST 'http://localhost:11434/api/copy' \ ---header 'Content-Type: text/plain' \ ---data-raw '{ +curl -X POST 'http://localhost:11434/api/copy' -d \ +'{ "source": "MyCoolModel", "destination": "ADifferentModel" }' @@ -301,9 +299,8 @@ No response is returned other than a 200 status code. #### Request ```shell -curl --location --request DELETE 'http://localhost:11434/api/delete' \ ---header 'Content-Type: text/plain' \ ---data-raw '{ +curl -X DELETE 'http://localhost:11434/api/delete' -d \ +'{ "name": "adifferentModel" }' ``` @@ -347,9 +344,8 @@ The response is a stream of JSON objects with the following format: #### Request ```shell -curl --location --request POST 'http://localhost:11434/api/pull' \ ---header 'Content-Type: text/plain' \ ---data-raw '{ +curl -X POST 'http://localhost:11434/api/pull' -d \ +'{ "name": "orca:3b-q4_1" }' ``` From 2b3bb41598232f83859c427d40a0bfac5fb70fc6 Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Mon, 7 Aug 2023 13:17:16 -0700 Subject: [PATCH 14/16] model name format added Signed-off-by: Matt Williams --- docs/api.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/docs/api.md b/docs/api.md index d0e0da9c..413a2e9f 100644 --- a/docs/api.md +++ b/docs/api.md @@ -9,13 +9,25 @@ - [Delete a Model](#delete-a-model) - [Pull a Model](#pull-a-model) +## Things to keep in mind when using the API + +### Model name format + +The model name format is `site/namespace/model:tag`. **Site** and **namespace** are optional, but will default to `registry.ollama.ai/library`. + +### Durations + +All durations are in nanoseconds. + + + ## Generate a Prompt **POST /api/generate** ### Description -**Generate** is the main endpoint that you will use when working with Ollama. This is used to generate a response to a prompt sent to a model. +**Generate** is the main endpoint that you will use when working with Ollama. This is used to generate a response to a prompt sent to a model. This is a streaming endpoint, so will be a series of responses. The final response will include the context and what is usually seen in the output from verbose mode. ### Request @@ -23,7 +35,7 @@ The **Generate** endpoint takes a JSON object with the following fields: ```JSON { - "model": "modelname", + "model": "site/namespace/model:tag", "prompt": "You are a software engineer working on building docs for Ollama.", "options": { "temperature": 0.7, From 13aace3d34e7ccc0c4848a9a31814f2673bb6ccd Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Mon, 7 Aug 2023 13:21:54 -0700 Subject: [PATCH 15/16] clarify some more Signed-off-by: Matt Williams --- docs/api.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/api.md b/docs/api.md index 413a2e9f..e3d20bb4 100644 --- a/docs/api.md +++ b/docs/api.md @@ -13,14 +13,12 @@ ### Model name format -The model name format is `site/namespace/model:tag`. **Site** and **namespace** are optional, but will default to `registry.ollama.ai/library`. +The model name format is `site/namespace/model:tag`. **Site** and **namespace** are optional, but will default to `registry.ollama.ai/library`. You will see this format used in `~/.ollama/models/manifests/`. ### Durations All durations are in nanoseconds. - - ## Generate a Prompt **POST /api/generate** From 0c52b4509bb176b797f6ba6411cd09390c4309f9 Mon Sep 17 00:00:00 2001 From: Matt Williams Date: Mon, 7 Aug 2023 13:27:58 -0700 Subject: [PATCH 16/16] get rid of namespace and site Signed-off-by: Matt Williams --- docs/api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/api.md b/docs/api.md index e3d20bb4..27eb4031 100644 --- a/docs/api.md +++ b/docs/api.md @@ -13,7 +13,7 @@ ### Model name format -The model name format is `site/namespace/model:tag`. **Site** and **namespace** are optional, but will default to `registry.ollama.ai/library`. You will see this format used in `~/.ollama/models/manifests/`. +The model name format today is model:tag`. Some examples are `orca:3b-q4_1` and `llama2:70b`. The tag is optional and if not provided will default to `latest`. The tag is used to identify a specific version. ### Durations