2023-06-27 16:08:52 +00:00
|
|
|
# Ollama
|
2023-06-22 16:45:31 +00:00
|
|
|
|
2023-06-27 16:44:12 +00:00
|
|
|
- Run models easily
|
2023-06-27 16:08:52 +00:00
|
|
|
- Download, manage and import models
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
## Install
|
2023-06-22 16:45:31 +00:00
|
|
|
|
|
|
|
```
|
2023-06-27 16:08:52 +00:00
|
|
|
pip install ollama
|
2023-06-22 16:45:31 +00:00
|
|
|
```
|
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
## Example quickstart
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
```python
|
|
|
|
import ollama
|
2023-06-27 16:48:50 +00:00
|
|
|
ollama.generate("./llama-7b-ggml.bin", "hi")
|
2023-06-25 17:08:03 +00:00
|
|
|
```
|
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
## Reference
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:51:36 +00:00
|
|
|
### `ollama.generate(model, message)`
|
2023-06-27 16:08:52 +00:00
|
|
|
|
2023-06-27 16:51:36 +00:00
|
|
|
Generate a completion
|
2023-06-27 16:08:52 +00:00
|
|
|
|
|
|
|
```python
|
2023-06-27 16:51:36 +00:00
|
|
|
ollama.generate("./llama-7b-ggml.bin", "hi")
|
2023-06-25 17:08:03 +00:00
|
|
|
```
|
|
|
|
|
2023-06-27 16:51:36 +00:00
|
|
|
### `ollama.load(model)`
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:51:36 +00:00
|
|
|
Load a model for generation
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
```python
|
2023-06-27 16:56:53 +00:00
|
|
|
ollama.load("model")
|
2023-06-25 17:08:03 +00:00
|
|
|
```
|
|
|
|
|
2023-06-27 16:51:36 +00:00
|
|
|
### `ollama.models()`
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:44:12 +00:00
|
|
|
List available local models
|
2023-06-27 16:08:52 +00:00
|
|
|
|
|
|
|
```
|
|
|
|
models = ollama.models()
|
2023-06-25 17:08:03 +00:00
|
|
|
```
|
|
|
|
|
2023-06-27 16:51:36 +00:00
|
|
|
### `ollama.serve()`
|
2023-06-25 17:10:15 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
Serve the ollama http server
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:51:36 +00:00
|
|
|
## Cooming Soon
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:56:53 +00:00
|
|
|
### `ollama.pull(model)`
|
2023-06-27 16:08:52 +00:00
|
|
|
|
2023-06-27 16:44:12 +00:00
|
|
|
Download a model
|
2023-06-27 16:08:52 +00:00
|
|
|
|
|
|
|
```python
|
|
|
|
ollama.pull("huggingface.co/thebloke/llama-7b-ggml")
|
2023-06-25 17:08:03 +00:00
|
|
|
```
|
|
|
|
|
2023-06-27 16:56:53 +00:00
|
|
|
### `ollama.import(filename)`
|
2023-06-27 16:08:52 +00:00
|
|
|
|
2023-06-27 16:44:12 +00:00
|
|
|
Import a model from a file
|
2023-06-27 16:08:52 +00:00
|
|
|
|
|
|
|
```python
|
|
|
|
ollama.import("./path/to/model")
|
|
|
|
```
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:51:36 +00:00
|
|
|
### `ollama.search("query")`
|
2023-06-25 18:29:26 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
Search for compatible models that Ollama can run
|
2023-06-25 18:29:26 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
```python
|
|
|
|
ollama.search("llama-7b")
|
|
|
|
```
|
2023-06-25 17:08:03 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
## Future CLI
|
2023-06-25 18:29:26 +00:00
|
|
|
|
2023-06-27 17:51:20 +00:00
|
|
|
In the future, there will be an `ollama` CLI for running models on servers, in containers or for local development environments.
|
2023-06-27 16:44:12 +00:00
|
|
|
|
2023-06-27 16:08:52 +00:00
|
|
|
```
|
2023-06-27 17:52:32 +00:00
|
|
|
ollama generate huggingface.co/thebloke/llama-7b-ggml "hi"
|
2023-06-27 16:44:12 +00:00
|
|
|
> Downloading [================> ] 66.67% (2/3) 30.2MB/s
|
2023-06-27 16:08:52 +00:00
|
|
|
```
|
2023-06-27 17:46:46 +00:00
|
|
|
|
|
|
|
## Documentation
|
|
|
|
|
|
|
|
- [Development](docs/development.md)
|