From 6559a5b48f33a2f876cbe7facd0c40ddfc282a0a Mon Sep 17 00:00:00 2001 From: Jeffrey Morgan Date: Wed, 5 Jul 2023 15:37:33 -0400 Subject: [PATCH] updated readme --- README.md | 85 +++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 64 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 9495a911..6092bbf6 100644 --- a/README.md +++ b/README.md @@ -1,41 +1,67 @@ +![ollama](https://github.com/jmorganca/ollama/assets/251292/961f99bb-251a-4eec-897d-1ba99997ad0f) + # Ollama -An easy, fast runtime for large language models, powered by `llama.cpp`. +Run large language models with `llama.cpp`. -> _Note: this project is a work in progress. Certain models that can be run with `ollama` are intended for research and/or non-commercial use only._ +> Note: certain models that can be run with this project are intended for research and/or non-commercial use only. + +### Features + +- Download and run popular large language models +- Switch between multiple models on the fly +- Hardware acceleration where available (Metal, CUDA) +- Fast inference server written in C++, powered by [llama.cpp](https://github.com/ggerganov/llama.cpp) +- REST API to use with your application (python, typescript SDKs coming soon) ## Install -Using `pip`: +- Download for macOS +- Download for Windows (coming soon) +- Docker: `docker run -p 8080:8080 ollama/ollama` -``` -pip install ollama -``` - -Using `docker`: - -``` -docker run ollama/ollama -``` +You can also build the [binary from source](#building). ## Quickstart -To run a model, use `ollama run`: +Run the model that started it all. ``` -ollama run orca-mini-3b +ollama run llama ``` -You can also run models from hugging face: +## Example models + +### πŸ’¬ Chat + +Have a conversation. ``` -ollama run huggingface.co/TheBloke/orca_mini_3B-GGML +ollama run vicuna "Why is the sky blue?" ``` -Or directly via downloaded model files: +### πŸ—ΊοΈ Instructions + +Ask questions. Get answers. ``` -ollama run ~/Downloads/orca-mini-13b.ggmlv3.q4_0.bin +ollama run orca "Write an email to my boss." +``` + +### πŸ‘©β€πŸ’» Code completion + +Sometimes you just need a little help writing code. + +``` +ollama run replit "Give me react code to render a button" +``` + +### πŸ“– Storytelling + +Venture into the unknown. + +``` +ollama run storyteller "Once upon a time" ``` ## Building @@ -45,7 +71,24 @@ go generate ./... go build . ``` -## Documentation +To run it start the server: -- [Development](docs/development.md) -- [Python SDK](docs/python.md) +``` +./ollama server & +``` + +Finally, run a model! + +``` +./ollama run ~/Downloads/vicuna-7b-v1.3.ggmlv3.q4_1.bin +``` + +## API Reference + +### `POST /completion` + +Complete a prompt + +``` +curl -X POST http://localhost:8080/completion -H 'Content-Type: application/json' -d '{"model": "/path/to/model", "prompt": "Once upon a time", "stream": true}' +```