baalajimaestro/ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
ollama/Dockerfile

218 lines
9.1 KiB
Text
Raw Normal View History

bump go version to 1.22.5 to fix security vulnerabilities
2024-07-17 21:44:19 +00:00
ARG GOLANG_VERSION=1.22.5
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CMAKE_VERSION=3.22.1
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
ARG CUDA_VERSION_11=11.3.1
Enable cuda v12 flags
2024-07-12 18:35:41 +00:00
ARG CUDA_V11_ARCHITECTURES="50;52;53;60;61;62;70;72;75;80;86"
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
ARG CUDA_VERSION_12=12.4.0
Enable cuda v12 flags
2024-07-12 18:35:41 +00:00
ARG CUDA_V12_ARCHITECTURES="60;61;62;70;72;75;80;86;87;89;90;90a"
Bump linux ROCm to 6.1.2
2024-07-15 22:10:22 +00:00
ARG ROCM_VERSION=6.1.2
replace dockerfile
2023-09-22 18:57:31 +00:00
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
# Copy the minimal context we need to run the generate scripts
FROM scratch AS llm-code
COPY .git .git
COPY .gitmodules .gitmodules
COPY llm llm
update cuda docker image
2023-09-08 20:41:49 +00:00
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
FROM --platform=linux/amd64 nvidia/cuda:$CUDA_VERSION_11-devel-centos7 AS cuda-11-build-amd64
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CMAKE_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} sh /rh_linux_deps.sh
ENV PATH /opt/rh/devtoolset-10/root/usr/bin:$PATH
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG CGO_CFLAGS
Enable cuda v12 flags
2024-07-12 18:35:41 +00:00
ARG CUDA_V11_ARCHITECTURES
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
ENV GOARCH amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
OLLAMA_SKIP_STATIC_GENERATE=1 \
OLLAMA_SKIP_CPU_GENERATE=1 \
Enable cuda v12 flags
2024-07-12 18:35:41 +00:00
CMAKE_CUDA_ARCHITECTURES="${CUDA_V11_ARCHITECTURES}" \
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
CUDA_VARIANT="_v11" \
bash gen_linux.sh
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
FROM --platform=linux/amd64 nvidia/cuda:$CUDA_VERSION_12-devel-centos7 AS cuda-12-build-amd64
ARG CMAKE_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} sh /rh_linux_deps.sh
ENV PATH /opt/rh/devtoolset-10/root/usr/bin:$PATH
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
ARG CGO_CFLAGS
Enable cuda v12 flags
2024-07-12 18:35:41 +00:00
ARG CUDA_V12_ARCHITECTURES
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
ENV GOARCH amd64
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 \
OLLAMA_SKIP_CPU_GENERATE=1 \
Enable cuda v12 flags
2024-07-12 18:35:41 +00:00
CMAKE_CUDA_ARCHITECTURES="${CUDA_V12_ARCHITECTURES}" \
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
CUDA_VARIANT="_v12" \
Enable cuda v12 flags
2024-07-12 18:35:41 +00:00
OLLAMA_CUSTOM_CUDA_DEFS="-DGGML_CUDA_USE_GRAPHS=on" \
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
bash gen_linux.sh
FROM --platform=linux/arm64 nvidia/cuda:$CUDA_VERSION_11-devel-rockylinux8 AS cuda-11-build-server-arm64
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CMAKE_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} sh /rh_linux_deps.sh
Revert "Switch arm cuda base image to centos 7" This reverts commit 5dacc1ebe88fbd4fea355d4c664fdfb9547f360e.
2024-03-26 02:01:11 +00:00
ENV PATH /opt/rh/gcc-toolset-10/root/usr/bin:$PATH
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG CGO_CFLAGS
Enable cuda v12 flags
2024-07-12 18:35:41 +00:00
ARG CUDA_V11_ARCHITECTURES
ENV GOARCH arm64
RUN OLLAMA_SKIP_STATIC_GENERATE=1 \
OLLAMA_SKIP_CPU_GENERATE=1 \
CMAKE_CUDA_ARCHITECTURES="${CUDA_V11_ARCHITECTURES}" \
CUDA_VARIANT="_v11" \
bash gen_linux.sh
FROM --platform=linux/arm64 nvidia/cuda:$CUDA_VERSION_12-devel-rockylinux8 AS cuda-12-build-server-arm64
ARG CMAKE_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} sh /rh_linux_deps.sh
ENV PATH /opt/rh/gcc-toolset-10/root/usr/bin:$PATH
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
ARG CGO_CFLAGS
ARG CUDA_V12_ARCHITECTURES
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
ENV GOARCH arm64
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
RUN --mount=type=cache,target=/root/.ccache \
Enable cuda v12 flags
2024-07-12 18:35:41 +00:00
OLLAMA_SKIP_STATIC_GENERATE=1 \
OLLAMA_SKIP_CPU_GENERATE=1 \
CMAKE_CUDA_ARCHITECTURES="${CUDA_V12_ARCHITECTURES}" \
CUDA_VARIANT="_v12" \
OLLAMA_CUSTOM_CUDA_DEFS="-DGGML_CUDA_USE_GRAPHS=on" \
bash gen_linux.sh
Add Jetson cuda variants for arm This adds new variants for arm64 specific to Jetson platforms
2024-05-31 04:54:07 +00:00
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Revamp ROCm support This refines where we extract the LLM libraries to by adding a new OLLAMA_HOME env var, that defaults to `~/.ollama` The logic was already idempotenent, so this should speed up startups after the first time a new release is deployed. It also cleans up after itself. We now build only a single ROCm version (latest major) on both windows and linux. Given the large size of ROCms tensor files, we split the dependency out. It's bundled into the installer on windows, and a separate download on windows. The linux install script is now smart and detects the presence of AMD GPUs and looks to see if rocm v6 is already present, and if not, then downloads our dependency tar file. For Linux discovery, we now use sysfs and check each GPU against what ROCm supports so we can degrade to CPU gracefully instead of having llama.cpp+rocm assert/crash on us. For Windows, we now use go's windows dynamic library loading logic to access the amdhip64.dll APIs to query the GPU information.
2024-02-16 01:15:09 +00:00
FROM --platform=linux/amd64 rocm/dev-centos-7:${ROCM_VERSION}-complete AS rocm-build-amd64
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CMAKE_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} sh /rh_linux_deps.sh
ENV PATH /opt/rh/devtoolset-10/root/usr/bin:$PATH
ENV LIBRARY_PATH /opt/amdgpu/lib64
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG CGO_CFLAGS
ARG AMDGPU_TARGETS
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
ENV GOARCH amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_SKIP_CPU_GENERATE=1 bash gen_linux.sh
Split rocm back out of bundle (#6432) We're over budget for github's maximum release artifact size with rocm + 2 cuda versions. This splits rocm back out as a discrete artifact, but keeps the layout so it can be extracted into the same location as the main bundle.
2024-08-20 14:26:38 +00:00
RUN mkdir -p ../../dist/linux-amd64-rocm/lib/ollama && \
(cd /opt/rocm/lib && tar cf - rocblas/library) | (cd ../../dist/linux-amd64-rocm/lib/ollama && tar xf - )
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
FROM --platform=linux/amd64 centos:7 AS cpu-builder-amd64
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CMAKE_VERSION
ARG GOLANG_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} GOLANG_VERSION=${GOLANG_VERSION} sh /rh_linux_deps.sh
ENV PATH /opt/rh/devtoolset-10/root/usr/bin:$PATH
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG OLLAMA_CUSTOM_CPU_DEFS
ARG CGO_CFLAGS
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
ENV GOARCH amd64
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
FROM --platform=linux/amd64 cpu-builder-amd64 AS static-build-amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_CPU_TARGET="static" bash gen_linux.sh
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
FROM --platform=linux/amd64 cpu-builder-amd64 AS cpu-build-amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_CPU_TARGET="cpu" bash gen_linux.sh
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
FROM --platform=linux/amd64 cpu-builder-amd64 AS cpu_avx-build-amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_CPU_TARGET="cpu_avx" bash gen_linux.sh
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
FROM --platform=linux/amd64 cpu-builder-amd64 AS cpu_avx2-build-amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_CPU_TARGET="cpu_avx2" bash gen_linux.sh
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Switch amd container image base to rocky 8 The centos 7 arm mirrors have disappeared due to the EOL 2 days ago, and the vault sed workaround which works for x86 doesn't work for arm.
2024-07-02 17:23:05 +00:00
FROM --platform=linux/arm64 rockylinux:8 AS cpu-builder-arm64
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CMAKE_VERSION
ARG GOLANG_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} GOLANG_VERSION=${GOLANG_VERSION} sh /rh_linux_deps.sh
Switch amd container image base to rocky 8 The centos 7 arm mirrors have disappeared due to the EOL 2 days ago, and the vault sed workaround which works for x86 doesn't work for arm.
2024-07-02 17:23:05 +00:00
ENV PATH /opt/rh/gcc-toolset-10/root/usr/bin:$PATH
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG OLLAMA_CUSTOM_CPU_DEFS
ARG CGO_CFLAGS
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
ENV GOARCH arm64
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
FROM --platform=linux/arm64 cpu-builder-arm64 AS static-build-arm64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_CPU_TARGET="static" bash gen_linux.sh
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
FROM --platform=linux/arm64 cpu-builder-arm64 AS cpu-build-arm64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_CPU_TARGET="cpu" bash gen_linux.sh
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
# Intermediate stage used for ./scripts/build_linux.sh
FROM --platform=linux/amd64 cpu-build-amd64 AS build-amd64
ENV CGO_ENABLED 1
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
WORKDIR /go/src/github.com/ollama/ollama
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
COPY . .
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
COPY --from=static-build-amd64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
COPY --from=cpu_avx-build-amd64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
COPY --from=cpu_avx2-build-amd64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
COPY --from=cuda-11-build-amd64 /go/src/github.com/ollama/ollama/dist/ dist/
COPY --from=cuda-11-build-amd64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
COPY --from=cuda-12-build-amd64 /go/src/github.com/ollama/ollama/dist/ dist/
COPY --from=cuda-12-build-amd64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
COPY --from=rocm-build-amd64 /go/src/github.com/ollama/ollama/dist/ dist/
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
COPY --from=rocm-build-amd64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG GOFLAGS
ARG CGO_CFLAGS
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
Adjust layout to bin+lib/ollama
2024-08-14 23:32:57 +00:00
go build -trimpath -o dist/linux-amd64/bin/ollama .
update dockerfile
2023-09-06 21:14:20 +00:00
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
# Intermediate stage used for ./scripts/build_linux.sh
FROM --platform=linux/arm64 cpu-build-arm64 AS build-arm64
ENV CGO_ENABLED 1
ARG GOLANG_VERSION
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
WORKDIR /go/src/github.com/ollama/ollama
update `Dockerfile`
2023-07-03 22:56:17 +00:00
COPY . .
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
COPY --from=static-build-arm64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
Add cuda v12 variant and selection logic Based on compute capability and driver version, pick v12 or v11 cuda variants.
2024-06-14 03:46:14 +00:00
COPY --from=cuda-11-build-server-arm64 /go/src/github.com/ollama/ollama/dist/ dist/
COPY --from=cuda-11-build-server-arm64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
Enable cuda v12 flags
2024-07-12 18:35:41 +00:00
COPY --from=cuda-12-build-server-arm64 /go/src/github.com/ollama/ollama/dist/ dist/
COPY --from=cuda-12-build-server-arm64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG GOFLAGS
ARG CGO_CFLAGS
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
Adjust layout to bin+lib/ollama
2024-08-14 23:32:57 +00:00
go build -trimpath -o dist/linux-arm64/bin/ollama .
# Strip out ROCm dependencies to keep the primary image lean
FROM --platform=linux/amd64 ubuntu:22.04 as amd64-libs-without-rocm
COPY --from=build-amd64 /go/src/github.com/ollama/ollama/dist/linux-amd64/lib/ /scratch/
RUN cd /scratch/ollama/ && rm -rf rocblas libamd* libdrm* libroc* libhip* libhsa*
update `Dockerfile`
2023-07-03 22:56:17 +00:00
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
# Runtime stages
Switch back to ubuntu base The size increase for rocm support in the standard image is problematic We'll revisit multiple tags for rocm support in a follow up PR.
2024-01-26 00:46:01 +00:00
FROM --platform=linux/amd64 ubuntu:22.04 as runtime-amd64
Adjust layout to bin+lib/ollama
2024-08-14 23:32:57 +00:00
COPY --from=amd64-libs-without-rocm /scratch/ /lib/
Switch back to ubuntu base The size increase for rocm support in the standard image is problematic We'll revisit multiple tags for rocm support in a follow up PR.
2024-01-26 00:46:01 +00:00
RUN apt-get update && apt-get install -y ca-certificates
Adjust layout to bin+lib/ollama
2024-08-14 23:32:57 +00:00
COPY --from=build-amd64 /go/src/github.com/ollama/ollama/dist/linux-amd64/bin/ /bin/
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
FROM --platform=linux/arm64 ubuntu:22.04 as runtime-arm64
Adjust layout to bin+lib/ollama
2024-08-14 23:32:57 +00:00
COPY --from=build-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64/lib/ /lib/
replace dockerfile
2023-09-22 18:57:31 +00:00
RUN apt-get update && apt-get install -y ca-certificates
Adjust layout to bin+lib/ollama
2024-08-14 23:32:57 +00:00
COPY --from=build-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64/bin/ /bin/
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Add back ROCm container support This adds ROCm support back as a discrete image.
2024-01-26 00:58:05 +00:00
# Radeon images are much larger so we keep it distinct from the CPU/CUDA image
Revamp ROCm support This refines where we extract the LLM libraries to by adding a new OLLAMA_HOME env var, that defaults to `~/.ollama` The logic was already idempotenent, so this should speed up startups after the first time a new release is deployed. It also cleans up after itself. We now build only a single ROCm version (latest major) on both windows and linux. Given the large size of ROCms tensor files, we split the dependency out. It's bundled into the installer on windows, and a separate download on windows. The linux install script is now smart and detects the presence of AMD GPUs and looks to see if rocm v6 is already present, and if not, then downloads our dependency tar file. For Linux discovery, we now use sysfs and check each GPU against what ROCm supports so we can degrade to CPU gracefully instead of having llama.cpp+rocm assert/crash on us. For Windows, we now use go's windows dynamic library loading logic to access the amdhip64.dll APIs to query the GPU information.
2024-02-16 01:15:09 +00:00
FROM --platform=linux/amd64 rocm/dev-centos-7:${ROCM_VERSION}-complete as runtime-rocm
Add back ROCm container support This adds ROCm support back as a discrete image.
2024-01-26 00:58:05 +00:00
RUN update-pciids
Adjust layout to bin+lib/ollama
2024-08-14 23:32:57 +00:00
COPY --from=build-amd64 /go/src/github.com/ollama/ollama/dist/linux-amd64/bin/ /bin/
RUN ln -s /opt/rocm/lib /lib/ollama
Add back ROCm container support This adds ROCm support back as a discrete image.
2024-01-26 00:58:05 +00:00
EXPOSE 11434
ENV OLLAMA_HOST 0.0.0.0
ENTRYPOINT ["/bin/ollama"]
CMD ["serve"]
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
FROM runtime-$TARGETARCH
docker: fix volume permission errors
2023-09-30 19:32:06 +00:00
EXPOSE 11434
ENV OLLAMA_HOST 0.0.0.0
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Add back ROCm container support This adds ROCm support back as a discrete image.
2024-01-26 00:58:05 +00:00
ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
docker: set PATH, LD_LIBRARY_PATH, and capabilities (#1336) * docker: set PATH, LD_LIBRARY_PATH, and capabilities * example: update k8s gpu manifest
2023-12-01 05:16:56 +00:00
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
Add env var so podman will map cuda GPUs Without this env var, podman's GPU logic doesn't map the GPU through
2024-02-29 16:43:08 +00:00
ENV NVIDIA_VISIBLE_DEVICES=all
docker: set PATH, LD_LIBRARY_PATH, and capabilities (#1336) * docker: set PATH, LD_LIBRARY_PATH, and capabilities * example: update k8s gpu manifest
2023-12-01 05:16:56 +00:00
update `Dockerfile`
2023-07-03 22:56:17 +00:00
ENTRYPOINT ["/bin/ollama"]
fix dockerfile
2023-07-06 20:34:14 +00:00
CMD ["serve"]
Reference in a new issue Copy permalink