From f9b2f999accab9d4cc3cca62df65aac908523e13 Mon Sep 17 00:00:00 2001 From: Jeffrey Morgan Date: Sun, 15 Oct 2023 02:23:03 -0400 Subject: [PATCH] update readme with `docker` setup and link to `import.md` --- README.md | 26 +++++++++++++++++++------- docs/faq.md | 20 ++++++++++++++++++-- docs/import.md | 2 +- docs/linux.md | 1 - 4 files changed, 38 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index d24d46fe..4af275ac 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,10 @@ Get up and running with large language models locally. [Download](https://ollama.ai/download/Ollama-darwin.zip) +### Windows + +Coming soon! + ### Linux & WSL2 ``` @@ -23,9 +27,13 @@ curl https://ollama.ai/install.sh | sh [Manual install instructions](https://github.com/jmorganca/ollama/blob/main/docs/linux.md) -### Windows +### Docker -coming soon +``` +docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama +``` + +For GPU support, use `--gpus=all`. See the Docker [image](https://hub.docker.com/r/ollama/ollama) for more information. ## Quickstart @@ -56,11 +64,11 @@ Here are some example open-source models that can be downloaded: ## Customize your own model -### Import from GGUF or GGML +### Import from GGUF -Ollama supports importing GGUF and GGML file formats in the Modelfile. This means if you have a model that is not in the Ollama library, you can create it, iterate on it, and upload it to the Ollama library to share with others when you are ready. +Ollama supports importing GGUF models in the Modelfile: -1. Create a file named Modelfile, and add a `FROM` instruction with the local filepath to the model you want to import. +1. Create a file named `Modelfile`, with a `FROM` instruction with the local filepath to the model you want to import. ``` FROM ./vicuna-33b.Q4_0.gguf @@ -69,15 +77,19 @@ Ollama supports importing GGUF and GGML file formats in the Modelfile. This mean 2. Create the model in Ollama ``` - ollama create name -f path_to_modelfile + ollama create example -f Modelfile ``` 3. Run the model ``` - ollama run name + ollama run example ``` +### Import from PyTorch or Safetensors + +See the [guide](docs/import.md) on importing models for more information. + ### Customize a prompt Models from the Ollama library can be customized with a prompt. The example diff --git a/docs/faq.md b/docs/faq.md index 9d369f1d..2840b898 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -1,5 +1,21 @@ # FAQ +## How can I view the logs? + +On macOS: + +``` +cat ~/.ollama/logs/server.log +``` + +On Linux: + +``` +journalctl -u ollama +``` + +If you're running `ollama serve` directly, the logs will be printed to the console. + ## How can I expose the Ollama server? ```bash @@ -14,5 +30,5 @@ OLLAMA_ORIGINS=http://192.168.1.1:*,https://example.com ollama serve ## Where are models stored? -* macOS: Raw model data is stored under `~/.ollama/models`. -* Linux: Raw model data is stored under `/usr/share/ollama/.ollama/models` +- macOS: Raw model data is stored under `~/.ollama/models`. +- Linux: Raw model data is stored under `/usr/share/ollama/.ollama/models` diff --git a/docs/import.md b/docs/import.md index 44c3f127..81b0f4ee 100644 --- a/docs/import.md +++ b/docs/import.md @@ -23,7 +23,7 @@ git clone https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 cd Mistral-7B-Instruct-v0.1 ``` -### Step 2: Convert and quantize (PyTorch and Safetensors) +### Step 2: Convert and quantize (for PyTorch and Safetensors) A [Docker image](https://hub.docker.com/r/ollama/quantize) with the tooling required to convert and quantize models is available. diff --git a/docs/linux.md b/docs/linux.md index 8ba8bc45..406fdae7 100644 --- a/docs/linux.md +++ b/docs/linux.md @@ -80,4 +80,3 @@ To view logs of Ollama running as a startup service, run: ```bash journalctl -u ollama ``` -