6c5ccb11f9
This refines where we extract the LLM libraries to by adding a new OLLAMA_HOME env var, that defaults to `~/.ollama` The logic was already idempotenent, so this should speed up startups after the first time a new release is deployed. It also cleans up after itself. We now build only a single ROCm version (latest major) on both windows and linux. Given the large size of ROCms tensor files, we split the dependency out. It's bundled into the installer on windows, and a separate download on windows. The linux install script is now smart and detects the presence of AMD GPUs and looks to see if rocm v6 is already present, and if not, then downloads our dependency tar file. For Linux discovery, we now use sysfs and check each GPU against what ROCm supports so we can degrade to CPU gracefully instead of having llama.cpp+rocm assert/crash on us. For Windows, we now use go's windows dynamic library loading logic to access the amdhip64.dll APIs to query the GPU information. |
||
|---|---|---|
| .. | ||
| tutorials | ||
| api.md | ||
| development.md | ||
| faq.md | ||
| import.md | ||
| linux.md | ||
| modelfile.md | ||
| openai.md | ||
| README.md | ||
| troubleshooting.md | ||
| tutorials.md | ||
| windows.md | ||
Documentation
To get started, see the project's quickstart.
Ollama is a tool for running AI models on your hardware. Many users will choose to use the Command Line Interface (CLI) to work with Ollama. Learn more about all the commands in the CLI in the Main Readme.
Use the RESTful API using any language, including Python, JavaScript, Typescript, Go, Rust, and many more. Learn more about using the API in the API Documentation.
Create new models or modify models already in the library using the Modelfile. Learn more about the Modelfile syntax in the Modelfile Documentation.
Import models using source model weights found on Hugging Face and similar sites by referring to the Import Documentation.
Installing on Linux in most cases is easy using the script on ollama.com/download. To get more detail about the install, including CUDA drivers, see the Linux Documentation.
Many of our users like the flexibility of using our official Docker Image. Learn more about using Docker with Ollama using the Docker Documentation.
It is easy to install on Linux and Mac, but many users will choose to build Ollama on their own. To do this, refer to the Development Documentation.
If encountering a problem with Ollama, the best place to start is the logs. Find more information about them here in the Troubleshooting Guide.
Finally for all the questions that don't fit anywhere else, there is the FAQ
Tutorials apply the documentation to tasks.
For working code examples of using Ollama, see Examples.