2023-06-27 16:08:52 +00:00
|
|
|
# Ollama
|
2023-06-22 16:45:31 +00:00
|
|
|
|
2023-06-28 13:57:36 +00:00
|
|
|
Run ai models locally.
|
2023-06-27 21:13:07 +00:00
|
|
|
|
2023-06-28 14:19:33 +00:00
|
|
|
> _Note: this project is a work in progress. The features below are still in development_
|
2023-06-27 21:13:07 +00:00
|
|
|
|
2023-06-28 13:57:36 +00:00
|
|
|
**Features**
|
2023-06-27 21:13:07 +00:00
|
|
|
|
2023-06-28 13:57:36 +00:00
|
|
|
- Run models locally on macOS (Windows, Linux and other platforms coming soon)
|
2023-06-28 14:19:07 +00:00
|
|
|
- Ollama uses the fastest loader available for your platform and model (e.g. llama.cpp, Core ML and other loaders coming soon)
|
2023-06-28 13:57:36 +00:00
|
|
|
- Import models from local files
|
|
|
|
- Find and download models on Hugging Face and other sources (coming soon)
|
|
|
|
- Support for running and switching between multiple models at a time (coming soon)
|
|
|
|
- Native desktop experience (coming soon)
|
|
|
|
- Built-in memory (coming soon)
|
|
|
|
|
|
|
|
## Install
|
2023-06-22 16:45:31 +00:00
|
|
|
|
|
|
|
```
|
2023-06-27 16:08:52 +00:00
|
|
|
pip install ollama
|
2023-06-22 16:45:31 +00:00
|
|
|
```
|
|
|
|
|
2023-06-28 13:57:36 +00:00
|
|
|
## Quickstart
|
2023-06-25 17:08:03 +00:00
|
|
|
|
|
|
|
```
|
2023-06-28 13:57:36 +00:00
|
|
|
% ollama run huggingface.co/TheBloke/orca_mini_3B-GGML
|
|
|
|
Pulling huggingface.co/TheBloke/orca_mini_3B-GGML...
|
2023-06-28 20:04:53 +00:00
|
|
|
Downloading [================ ] 66.67% 11.8MiB/s
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-28 13:57:36 +00:00
|
|
|
...
|
|
|
|
...
|
|
|
|
...
|
2023-06-27 16:08:52 +00:00
|
|
|
|
2023-06-28 13:57:36 +00:00
|
|
|
> Hello
|
|
|
|
|
|
|
|
Hello, how may I help you?
|
|
|
|
```
|
|
|
|
|
|
|
|
## Python SDK
|
|
|
|
|
|
|
|
### Example
|
2023-06-27 16:08:52 +00:00
|
|
|
|
|
|
|
```python
|
2023-06-28 13:57:36 +00:00
|
|
|
import ollama
|
2023-06-28 14:19:07 +00:00
|
|
|
ollama.generate("orca-mini-3b", "hi")
|
2023-06-25 17:08:03 +00:00
|
|
|
```
|
|
|
|
|
2023-06-28 13:57:36 +00:00
|
|
|
### `ollama.generate(model, message)`
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-28 13:57:36 +00:00
|
|
|
Generate a completion
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
```python
|
2023-06-28 13:57:36 +00:00
|
|
|
ollama.generate("./llama-7b-ggml.bin", "hi")
|
2023-06-25 17:08:03 +00:00
|
|
|
```
|
|
|
|
|
2023-06-27 16:51:36 +00:00
|
|
|
### `ollama.models()`
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:44:12 +00:00
|
|
|
List available local models
|
2023-06-27 16:08:52 +00:00
|
|
|
|
2023-06-28 14:57:18 +00:00
|
|
|
```python
|
2023-06-27 16:08:52 +00:00
|
|
|
models = ollama.models()
|
2023-06-25 17:08:03 +00:00
|
|
|
```
|
|
|
|
|
2023-06-27 16:51:36 +00:00
|
|
|
### `ollama.serve()`
|
2023-06-25 17:10:15 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
Serve the ollama http server
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-28 14:57:18 +00:00
|
|
|
```
|
|
|
|
ollama.serve()
|
|
|
|
```
|
|
|
|
|
2023-06-27 21:36:02 +00:00
|
|
|
### `ollama.add(filepath)`
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 21:36:02 +00:00
|
|
|
Add a model by importing from a file
|
2023-06-27 16:08:52 +00:00
|
|
|
|
|
|
|
```python
|
2023-06-27 21:36:02 +00:00
|
|
|
ollama.add("./path/to/model")
|
2023-06-25 17:08:03 +00:00
|
|
|
```
|
|
|
|
|
2023-06-28 13:57:36 +00:00
|
|
|
### `ollama.load(model)`
|
|
|
|
|
|
|
|
Manually a model for generation
|
|
|
|
|
|
|
|
```python
|
|
|
|
ollama.load("model")
|
|
|
|
```
|
|
|
|
|
|
|
|
### `ollama.unload(model)`
|
|
|
|
|
|
|
|
Unload a model
|
|
|
|
|
|
|
|
```python
|
|
|
|
ollama.unload("model")
|
|
|
|
```
|
|
|
|
|
2023-06-27 21:36:02 +00:00
|
|
|
### `ollama.pull(model)`
|
|
|
|
|
|
|
|
Download a model
|
2023-06-27 16:08:52 +00:00
|
|
|
|
|
|
|
```python
|
2023-06-27 21:36:02 +00:00
|
|
|
ollama.pull("huggingface.co/thebloke/llama-7b-ggml")
|
2023-06-27 16:08:52 +00:00
|
|
|
```
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-28 18:39:43 +00:00
|
|
|
## Coming Soon
|
2023-06-28 16:13:13 +00:00
|
|
|
|
2023-06-27 16:51:36 +00:00
|
|
|
### `ollama.search("query")`
|
2023-06-25 18:29:26 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
Search for compatible models that Ollama can run
|
2023-06-25 18:29:26 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
```python
|
|
|
|
ollama.search("llama-7b")
|
|
|
|
```
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 17:46:46 +00:00
|
|
|
## Documentation
|
|
|
|
|
|
|
|
- [Development](docs/development.md)
|