2023-07-18 19:45:38 +00:00
< div align = "center" >
< picture >
2023-07-20 15:55:20 +00:00
< source media = "(prefers-color-scheme: dark)" height = "200px" srcset = "https://github.com/jmorganca/ollama/assets/3325447/56ea1849-1284-4645-8970-956de6e51c3c" >
< img alt = "logo" height = "200px" src = "https://github.com/jmorganca/ollama/assets/3325447/0d0b44e2-8f4a-4e99-9b52-a5c1c741c8f7" >
2023-07-18 19:45:38 +00:00
< / picture >
< / div >
2023-07-05 19:37:33 +00:00
2023-06-27 16:08:52 +00:00
# Ollama
2023-06-22 16:45:31 +00:00
2023-07-19 19:31:48 +00:00
[![Discord ](https://dcbadge.vercel.app/api/server/ollama?style=flat&compact=true )](https://discord.gg/ollama)
2023-07-19 19:28:50 +00:00
2023-07-20 19:21:29 +00:00
Run, create, and share large language models (LLMs).
2023-07-20 15:33:28 +00:00
2023-08-08 22:50:23 +00:00
> Note: Ollama is in early preview. Please report any issues you find.
2023-07-18 20:31:25 +00:00
## Download
2023-08-04 19:45:55 +00:00
- [Download ](https://ollama.ai/download ) for macOS
2023-07-18 20:31:25 +00:00
- Download for Windows and Linux (coming soon)
- Build [from source ](#building )
2023-07-19 19:28:50 +00:00
## Quickstart
To run and chat with [Llama 2 ](https://ai.meta.com/llama ), the new model by Meta:
```
ollama run llama2
```
## Model library
2023-08-26 03:44:26 +00:00
Ollama supports a list of open-source models available on [ollama.ai/library ](https://ollama.ai/library 'ollama model library' )
2023-08-17 02:53:27 +00:00
2023-08-26 03:44:26 +00:00
Here are some example open-source models that can be downloaded:
2023-07-19 19:28:50 +00:00
2023-08-01 15:25:01 +00:00
| Model | Parameters | Size | Download |
| ------------------------ | ---------- | ----- | ------------------------------- |
| Llama2 | 7B | 3.8GB | `ollama pull llama2` |
| Llama2 13B | 13B | 7.3GB | `ollama pull llama2:13b` |
2023-08-08 20:08:05 +00:00
| Llama2 70B | 70B | 39GB | `ollama pull llama2:70b` |
| Llama2 Uncensored | 7B | 3.8GB | `ollama pull llama2-uncensored` |
2023-08-26 03:44:26 +00:00
| Code Llama | 7B | 3.8GB | `ollama pull codellama` |
2023-08-16 01:10:28 +00:00
| Orca Mini | 3B | 1.9GB | `ollama pull orca-mini` |
2023-08-01 15:25:01 +00:00
| Vicuna | 7B | 3.8GB | `ollama pull vicuna` |
2023-08-17 03:42:00 +00:00
| Nous-Hermes | 7B | 3.8GB | `ollama pull nous-hermes` |
| Nous-Hermes 13B | 13B | 7.3GB | `ollama pull nous-hermes:13b` |
2023-08-01 15:25:01 +00:00
| Wizard Vicuna Uncensored | 13B | 7.3GB | `ollama pull wizard-vicuna` |
2023-07-19 19:28:50 +00:00
2023-07-20 19:21:29 +00:00
> Note: You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models.
2023-07-18 20:22:33 +00:00
## Examples
2023-06-27 21:13:07 +00:00
2023-09-01 14:54:31 +00:00
### Pull a public model
```
ollama pull llama2
```
> This command can also be used to update a local model. Only updated changes will be pulled.
2023-09-01 20:44:14 +00:00
### Run a model interactively
2023-06-30 16:39:25 +00:00
2023-06-22 16:45:31 +00:00
```
2023-07-18 20:22:33 +00:00
ollama run llama2
>>> hi
Hello! How can I help you today?
2023-06-22 16:45:31 +00:00
```
2023-08-10 16:54:03 +00:00
For multiline input, you can wrap text with `"""` :
2023-08-10 15:22:28 +00:00
```
>>> """Hello,
... world!
... """
I'm a basic program that prints the famous "Hello, world!" message to the console.
```
2023-09-01 20:44:14 +00:00
### Run a model non-interactively
```
$ ollama run llama2 'tell me a joke'
Sure! Here's a quick one:
Why did the scarecrow win an award? Because he was outstanding in his field!
```
```
$ cat < < EOF > prompts.txt
tell me a joke about llamas
tell me another one
EOF
$ ollama run llama2 < prompts.txt
>>> tell me a joke about llamas
Why did the llama refuse to play hide-and-seek?
nobody likes to be hided!
>>> tell me another one
Sure, here's another one:
Why did the llama go to the bar?
To have a hay-often good time!
```
### Run a model on contents of a text file
```
$ ollama run llama2 "summarize this file:" "$(cat README.md)"
Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications.
```
2023-09-01 14:54:31 +00:00
### Customize a model
2023-07-19 19:28:50 +00:00
Pull a base model:
```
2023-07-20 09:21:51 +00:00
ollama pull llama2
2023-07-19 19:28:50 +00:00
```
2023-08-08 22:41:48 +00:00
2023-07-18 20:22:33 +00:00
Create a `Modelfile` :
2023-07-05 19:37:33 +00:00
2023-06-30 16:31:00 +00:00
```
2023-07-20 09:21:51 +00:00
FROM llama2
2023-07-20 15:17:09 +00:00
# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1
# set the system prompt
2023-07-20 09:21:51 +00:00
SYSTEM """
2023-07-18 20:32:06 +00:00
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
2023-07-18 20:22:33 +00:00
"""
2023-06-29 22:25:02 +00:00
```
2023-07-07 20:14:58 +00:00
2023-07-18 20:22:33 +00:00
Next, create and run the model:
2023-07-07 20:14:58 +00:00
```
2023-07-18 20:22:33 +00:00
ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.
2023-07-07 20:14:58 +00:00
```
2023-08-10 16:54:03 +00:00
For more examples, see the [examples ](./examples ) directory. For more information on creating a Modelfile, see the [Modelfile ](./docs/modelfile.md ) documentation.
2023-07-19 19:28:50 +00:00
2023-09-01 14:54:31 +00:00
### Listing local models
2023-07-06 20:21:01 +00:00
2023-07-19 19:28:50 +00:00
```
2023-09-01 14:54:31 +00:00
ollama list
2023-07-19 19:28:50 +00:00
```
2023-06-28 13:57:36 +00:00
2023-09-01 14:54:31 +00:00
### Removing local models
2023-07-20 19:21:29 +00:00
```
2023-09-01 14:54:31 +00:00
ollama rm llama2
2023-07-20 19:21:29 +00:00
```
## Model packages
### Overview
2023-09-01 14:54:31 +00:00
Ollama bundles model weights, configurations, and data into a single package, defined by a [Modelfile ](./docs/modelfile.md ).
2023-07-20 19:21:29 +00:00
< picture >
< source media = "(prefers-color-scheme: dark)" height = "480" srcset = "https://github.com/jmorganca/ollama/assets/251292/2fd96b5f-191b-45c1-9668-941cfad4eb70" >
< img alt = "logo" height = "480" src = "https://github.com/jmorganca/ollama/assets/251292/2fd96b5f-191b-45c1-9668-941cfad4eb70" >
< / picture >
2023-07-03 20:32:48 +00:00
## Building
2023-09-07 10:43:26 +00:00
Install `cmake` and `go` :
2023-08-25 18:44:25 +00:00
2023-07-03 20:32:48 +00:00
```
2023-08-30 21:54:02 +00:00
brew install cmake
2023-09-07 10:43:26 +00:00
brew install go
2023-08-30 21:54:02 +00:00
```
Then generate dependencies and build:
```
go generate ./...
2023-07-11 16:50:02 +00:00
go build .
2023-07-03 20:32:48 +00:00
```
2023-08-30 21:54:02 +00:00
Next, start the server:
2023-06-27 17:46:46 +00:00
2023-07-05 19:37:33 +00:00
```
2023-08-30 21:54:02 +00:00
./ollama serve
2023-07-05 19:37:33 +00:00
```
2023-09-01 14:54:31 +00:00
Finally, in a separate shell, run a model:
2023-07-05 19:37:33 +00:00
```
2023-07-18 20:22:33 +00:00
./ollama run llama2
2023-07-05 19:37:33 +00:00
```
2023-07-21 07:47:17 +00:00
## REST API
2023-08-08 22:48:47 +00:00
> See the [API documentation](./docs/api.md) for all endpoints.
2023-07-21 07:47:17 +00:00
2023-08-08 22:48:47 +00:00
Ollama has an API for running and managing models. For example to generate text from a model:
2023-07-21 07:47:17 +00:00
```
2023-08-08 22:48:47 +00:00
curl -X POST http://localhost:11434/api/generate -d '{
"model": "llama2",
"prompt":"Why is the sky blue?"
}'
2023-07-23 22:01:05 +00:00
```
2023-07-31 20:59:39 +00:00
2023-09-01 20:44:14 +00:00
## Community Projects using Ollama
2023-07-31 20:59:39 +00:00
2023-08-09 04:03:10 +00:00
- [LangChain ](https://python.langchain.com/docs/integrations/llms/ollama ) and [LangChain.js ](https://js.langchain.com/docs/modules/model_io/models/llms/integrations/ollama ) with a question-answering [example ](https://js.langchain.com/docs/use_cases/question_answering/local_retrieval_qa ).
2023-08-01 15:14:54 +00:00
- [Continue ](https://github.com/continuedev/continue ) - embeds Ollama inside Visual Studio Code. The extension lets you highlight code to add to the prompt, ask questions in the sidebar, and generate code inline.
2023-08-13 00:47:53 +00:00
- [LiteLLM ](https://github.com/BerriAI/litellm ) a lightweight python package to simplify LLM API calls
2023-08-01 15:14:54 +00:00
- [Discord AI Bot ](https://github.com/mekb-turtle/discord-ai-bot ) - interact with Ollama as a chatbot on Discord.
2023-08-01 15:25:01 +00:00
- [Raycast Ollama ](https://github.com/MassimilianoPasquini97/raycast_ollama ) - Raycast extension to use Ollama for local llama inference on Raycast.
2023-08-03 15:22:57 +00:00
- [Simple HTML UI for Ollama ](https://github.com/rtcfirefly/ollama-ui )
2023-08-11 03:13:47 +00:00
- [Emacs client ](https://github.com/zweifisch/ollama ) for Ollama