ollama/README.md

148 lines
3.5 KiB
Markdown
Raw Normal View History

2023-07-18 19:45:38 +00:00
<div align="center">
<picture>
2023-07-20 15:55:20 +00:00
<source media="(prefers-color-scheme: dark)" height="200px" srcset="https://github.com/jmorganca/ollama/assets/3325447/56ea1849-1284-4645-8970-956de6e51c3c">
<img alt="logo" height="200px" src="https://github.com/jmorganca/ollama/assets/3325447/0d0b44e2-8f4a-4e99-9b52-a5c1c741c8f7">
2023-07-18 19:45:38 +00:00
</picture>
</div>
2023-07-05 19:37:33 +00:00
2023-06-27 16:08:52 +00:00
# Ollama
2023-06-22 16:45:31 +00:00
2023-07-19 19:31:48 +00:00
[![Discord](https://dcbadge.vercel.app/api/server/ollama?style=flat&compact=true)](https://discord.gg/ollama)
2023-07-19 19:28:50 +00:00
2023-07-18 20:22:33 +00:00
> Note: Ollama is in early preview. Please report any issues you find.
2023-07-05 19:37:33 +00:00
2023-07-20 19:21:29 +00:00
Run, create, and share large language models (LLMs).
2023-07-20 15:33:28 +00:00
## Download
- [Download](https://ollama.ai/download) for macOS on Apple Silicon (Intel coming soon)
- Download for Windows and Linux (coming soon)
- Build [from source](#building)
2023-07-19 19:28:50 +00:00
## Quickstart
To run and chat with [Llama 2](https://ai.meta.com/llama), the new model by Meta:
```
ollama run llama2
```
## Model library
2023-07-20 19:21:29 +00:00
`ollama` includes a library of open-source models:
2023-07-19 19:28:50 +00:00
| Model | Parameters | Size | Download |
| ------------------------ | ---------- | ----- | --------------------------- |
| Llama2 | 7B | 3.8GB | `ollama pull llama2` |
| Llama2 13B | 13B | 7.3GB | `ollama pull llama2:13b` |
| Orca Mini | 3B | 1.9GB | `ollama pull orca` |
| Vicuna | 7B | 3.8GB | `ollama pull vicuna` |
| Nous-Hermes | 13B | 7.3GB | `ollama pull nous-hermes` |
| Wizard Vicuna Uncensored | 13B | 7.3GB | `ollama pull wizard-vicuna` |
2023-07-20 19:21:29 +00:00
> Note: You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models.
2023-07-18 20:22:33 +00:00
## Examples
2023-06-27 21:13:07 +00:00
2023-07-19 19:28:50 +00:00
### Run a model
2023-06-22 16:45:31 +00:00
```
2023-07-18 20:22:33 +00:00
ollama run llama2
>>> hi
Hello! How can I help you today?
2023-06-22 16:45:31 +00:00
```
2023-07-20 19:21:29 +00:00
### Create a custom model
2023-07-19 19:28:50 +00:00
Pull a base model:
```
2023-07-20 09:21:51 +00:00
ollama pull llama2
2023-07-19 19:28:50 +00:00
```
2023-07-05 19:37:33 +00:00
2023-07-18 20:22:33 +00:00
Create a `Modelfile`:
2023-07-05 19:37:33 +00:00
2023-06-30 16:31:00 +00:00
```
2023-07-20 09:21:51 +00:00
FROM llama2
2023-07-20 15:17:09 +00:00
# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1
# set the system prompt
2023-07-20 09:21:51 +00:00
SYSTEM """
2023-07-18 20:32:06 +00:00
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
2023-07-18 20:22:33 +00:00
"""
2023-06-29 22:25:02 +00:00
```
2023-07-18 20:22:33 +00:00
Next, create and run the model:
```
2023-07-18 20:22:33 +00:00
ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.
```
2023-07-20 09:15:11 +00:00
For more examples, see the [examples](./examples) directory.
2023-07-27 19:13:50 +00:00
2023-07-27 19:13:04 +00:00
For more information on creating a Modelfile, see the [Modelfile](./docs/modelfile.md) documentation.
2023-07-19 19:28:50 +00:00
### Pull a model from the registry
2023-07-06 20:21:01 +00:00
2023-07-19 19:28:50 +00:00
```
2023-07-20 09:21:51 +00:00
ollama pull orca
2023-07-19 19:28:50 +00:00
```
2023-06-28 13:57:36 +00:00
2023-07-20 19:21:29 +00:00
### Listing local models
```
ollama list
```
## Model packages
### Overview
Ollama bundles model weights, configuration, and data into a single package, defined by a [Modelfile](./docs/modelfile.md).
<picture>
<source media="(prefers-color-scheme: dark)" height="480" srcset="https://github.com/jmorganca/ollama/assets/251292/2fd96b5f-191b-45c1-9668-941cfad4eb70">
<img alt="logo" height="480" src="https://github.com/jmorganca/ollama/assets/251292/2fd96b5f-191b-45c1-9668-941cfad4eb70">
</picture>
2023-07-03 20:32:48 +00:00
## Building
```
2023-07-11 16:50:02 +00:00
go build .
2023-07-03 20:32:48 +00:00
```
2023-07-05 19:37:33 +00:00
To run it start the server:
2023-06-27 17:46:46 +00:00
2023-07-05 19:37:33 +00:00
```
./ollama serve &
2023-07-05 19:37:33 +00:00
```
Finally, run a model!
```
2023-07-18 20:22:33 +00:00
./ollama run llama2
2023-07-05 19:37:33 +00:00
```
2023-07-21 07:47:17 +00:00
## REST API
### `POST /api/generate`
Generate text from a model.
```
curl -X POST http://localhost:11434/api/generate -d '{"model": "llama2", "prompt":"Why is the sky blue?"}'
```
2023-07-23 22:01:05 +00:00
### `POST /api/create`
Create a model from a `Modelfile`.
```
curl -X POST http://localhost:11434/api/create -d '{"name": "my-model", "path": "/path/to/modelfile"}'
```