2023-06-27 16:08:52 +00:00
|
|
|
# Ollama
|
2023-06-22 16:45:31 +00:00
|
|
|
|
2023-07-01 21:54:29 +00:00
|
|
|
Ollama is a tool for running large language models. It's designed to be easy to use and fast.
|
2023-06-27 21:13:07 +00:00
|
|
|
|
2023-06-30 15:39:24 +00:00
|
|
|
> _Note: this project is a work in progress. Certain models that can be run with `ollama` are intended for research and/or non-commercial use only._
|
2023-06-28 13:57:36 +00:00
|
|
|
|
|
|
|
|
## Install
|
2023-06-22 16:45:31 +00:00
|
|
|
|
2023-06-30 16:39:25 +00:00
|
|
|
Using `pip`:
|
|
|
|
|
|
2023-06-22 16:45:31 +00:00
|
|
|
```
|
2023-06-27 16:08:52 +00:00
|
|
|
pip install ollama
|
2023-06-22 16:45:31 +00:00
|
|
|
```
|
|
|
|
|
|
2023-06-30 16:39:25 +00:00
|
|
|
Using `docker`:
|
|
|
|
|
|
2023-06-30 16:31:00 +00:00
|
|
|
```
|
|
|
|
|
docker run ollama/ollama
|
|
|
|
|
```
|
|
|
|
|
|
2023-06-28 13:57:36 +00:00
|
|
|
## Quickstart
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-29 22:25:02 +00:00
|
|
|
To run a model, use `ollama run`:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
ollama run orca-mini-3b
|
2023-06-25 17:08:03 +00:00
|
|
|
```
|
|
|
|
|
|
2023-06-29 22:25:02 +00:00
|
|
|
You can also run models from hugging face:
|
2023-06-27 16:08:52 +00:00
|
|
|
|
2023-06-29 22:25:02 +00:00
|
|
|
```
|
|
|
|
|
ollama run huggingface.co/TheBloke/orca_mini_3B-GGML
|
|
|
|
|
```
|
2023-06-28 13:57:36 +00:00
|
|
|
|
2023-06-29 22:25:02 +00:00
|
|
|
Or directly via downloaded model files:
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
ollama run ~/Downloads/orca-mini-13b.ggmlv3.q4_0.bin
|
2023-06-28 13:57:36 +00:00
|
|
|
```
|
|
|
|
|
|
2023-06-27 17:46:46 +00:00
|
|
|
## Documentation
|
|
|
|
|
|
|
|
|
|
- [Development](docs/development.md)
|
2023-07-01 21:54:29 +00:00
|
|
|
- [Python SDK](docs/python.md)
|
