History

Jeffrey Morgan 1c8435ffa9 Update domain name references in docs and install script (#2435 )		2024-02-09 15:19:30 -08:00
..
cpu.yaml	k8s example	2023-11-01 14:52:58 -07:00
gpu.yaml	docker: set PATH, LD_LIBRARY_PATH, and capabilities (#1336 )	2023-11-30 21:16:56 -08:00
README.md	Update domain name references in docs and install script (#2435 )	2024-02-09 15:19:30 -08:00

Deploy Ollama to Kubernetes

Prerequisites

Create the Ollama namespace, daemon set, and service
```
kubectl apply -f cpu.yaml
```
Port forward the Ollama service to connect and use it locally
```
kubectl -n ollama port-forward service/ollama 11434:80
```
Pull and run a model, for example orca-mini:3b
```
ollama run orca-mini:3b
```

Hardware acceleration in Kubernetes requires NVIDIA's k8s-device-plugin. Follow the link for more details.

Once configured, create a GPU enabled Ollama deployment.

kubectl apply -f gpu.yaml