update readme with docker
setup and link to import.md
This commit is contained in:
parent
c416087339
commit
f9b2f999ac
4 changed files with 38 additions and 11 deletions
26
README.md
26
README.md
|
@ -15,6 +15,10 @@ Get up and running with large language models locally.
|
||||||
|
|
||||||
[Download](https://ollama.ai/download/Ollama-darwin.zip)
|
[Download](https://ollama.ai/download/Ollama-darwin.zip)
|
||||||
|
|
||||||
|
### Windows
|
||||||
|
|
||||||
|
Coming soon!
|
||||||
|
|
||||||
### Linux & WSL2
|
### Linux & WSL2
|
||||||
|
|
||||||
```
|
```
|
||||||
|
@ -23,9 +27,13 @@ curl https://ollama.ai/install.sh | sh
|
||||||
|
|
||||||
[Manual install instructions](https://github.com/jmorganca/ollama/blob/main/docs/linux.md)
|
[Manual install instructions](https://github.com/jmorganca/ollama/blob/main/docs/linux.md)
|
||||||
|
|
||||||
### Windows
|
### Docker
|
||||||
|
|
||||||
coming soon
|
```
|
||||||
|
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
|
||||||
|
```
|
||||||
|
|
||||||
|
For GPU support, use `--gpus=all`. See the Docker [image](https://hub.docker.com/r/ollama/ollama) for more information.
|
||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
|
@ -56,11 +64,11 @@ Here are some example open-source models that can be downloaded:
|
||||||
|
|
||||||
## Customize your own model
|
## Customize your own model
|
||||||
|
|
||||||
### Import from GGUF or GGML
|
### Import from GGUF
|
||||||
|
|
||||||
Ollama supports importing GGUF and GGML file formats in the Modelfile. This means if you have a model that is not in the Ollama library, you can create it, iterate on it, and upload it to the Ollama library to share with others when you are ready.
|
Ollama supports importing GGUF models in the Modelfile:
|
||||||
|
|
||||||
1. Create a file named Modelfile, and add a `FROM` instruction with the local filepath to the model you want to import.
|
1. Create a file named `Modelfile`, with a `FROM` instruction with the local filepath to the model you want to import.
|
||||||
|
|
||||||
```
|
```
|
||||||
FROM ./vicuna-33b.Q4_0.gguf
|
FROM ./vicuna-33b.Q4_0.gguf
|
||||||
|
@ -69,15 +77,19 @@ Ollama supports importing GGUF and GGML file formats in the Modelfile. This mean
|
||||||
2. Create the model in Ollama
|
2. Create the model in Ollama
|
||||||
|
|
||||||
```
|
```
|
||||||
ollama create name -f path_to_modelfile
|
ollama create example -f Modelfile
|
||||||
```
|
```
|
||||||
|
|
||||||
3. Run the model
|
3. Run the model
|
||||||
|
|
||||||
```
|
```
|
||||||
ollama run name
|
ollama run example
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Import from PyTorch or Safetensors
|
||||||
|
|
||||||
|
See the [guide](docs/import.md) on importing models for more information.
|
||||||
|
|
||||||
### Customize a prompt
|
### Customize a prompt
|
||||||
|
|
||||||
Models from the Ollama library can be customized with a prompt. The example
|
Models from the Ollama library can be customized with a prompt. The example
|
||||||
|
|
20
docs/faq.md
20
docs/faq.md
|
@ -1,5 +1,21 @@
|
||||||
# FAQ
|
# FAQ
|
||||||
|
|
||||||
|
## How can I view the logs?
|
||||||
|
|
||||||
|
On macOS:
|
||||||
|
|
||||||
|
```
|
||||||
|
cat ~/.ollama/logs/server.log
|
||||||
|
```
|
||||||
|
|
||||||
|
On Linux:
|
||||||
|
|
||||||
|
```
|
||||||
|
journalctl -u ollama
|
||||||
|
```
|
||||||
|
|
||||||
|
If you're running `ollama serve` directly, the logs will be printed to the console.
|
||||||
|
|
||||||
## How can I expose the Ollama server?
|
## How can I expose the Ollama server?
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
@ -14,5 +30,5 @@ OLLAMA_ORIGINS=http://192.168.1.1:*,https://example.com ollama serve
|
||||||
|
|
||||||
## Where are models stored?
|
## Where are models stored?
|
||||||
|
|
||||||
* macOS: Raw model data is stored under `~/.ollama/models`.
|
- macOS: Raw model data is stored under `~/.ollama/models`.
|
||||||
* Linux: Raw model data is stored under `/usr/share/ollama/.ollama/models`
|
- Linux: Raw model data is stored under `/usr/share/ollama/.ollama/models`
|
||||||
|
|
|
@ -23,7 +23,7 @@ git clone https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1
|
||||||
cd Mistral-7B-Instruct-v0.1
|
cd Mistral-7B-Instruct-v0.1
|
||||||
```
|
```
|
||||||
|
|
||||||
### Step 2: Convert and quantize (PyTorch and Safetensors)
|
### Step 2: Convert and quantize (for PyTorch and Safetensors)
|
||||||
|
|
||||||
A [Docker image](https://hub.docker.com/r/ollama/quantize) with the tooling required to convert and quantize models is available.
|
A [Docker image](https://hub.docker.com/r/ollama/quantize) with the tooling required to convert and quantize models is available.
|
||||||
|
|
||||||
|
|
|
@ -80,4 +80,3 @@ To view logs of Ollama running as a startup service, run:
|
||||||
```bash
|
```bash
|
||||||
journalctl -u ollama
|
journalctl -u ollama
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
Loading…
Reference in a new issue