baalajimaestro/ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
ollama/Dockerfile

199 lines
8.6 KiB
Text
Raw Normal View History

bump go version to 1.22.5 to fix security vulnerabilities
2024-07-17 21:44:19 +00:00
ARG GOLANG_VERSION=1.22.5
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CMAKE_VERSION=3.22.1
doc: faq gpu compatibility (#3142)
2024-03-21 09:21:34 +00:00
# this CUDA_VERSION corresponds with the one specified in docs/gpu.md
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CUDA_VERSION=11.3.1
Bump linux ROCm to 6.1.2
2024-07-15 22:10:22 +00:00
ARG ROCM_VERSION=6.1.2
Add Jetson cuda variants for arm This adds new variants for arm64 specific to Jetson platforms
2024-05-31 04:54:07 +00:00
ARG JETPACK_6=r36.2.0
ARG JETPACK_5=r35.4.1
ARG JETPACK_4=r32.7.1
replace dockerfile
2023-09-22 18:57:31 +00:00
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
# Copy the minimal context we need to run the generate scripts
FROM scratch AS llm-code
COPY .git .git
COPY .gitmodules .gitmodules
COPY llm llm
update cuda docker image
2023-09-08 20:41:49 +00:00
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
FROM --platform=linux/amd64 nvidia/cuda:$CUDA_VERSION-devel-centos7 AS cuda-build-amd64
ARG CMAKE_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} sh /rh_linux_deps.sh
ENV PATH /opt/rh/devtoolset-10/root/usr/bin:$PATH
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG CGO_CFLAGS
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
ENV GOARCH amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_SKIP_CPU_GENERATE=1 bash gen_linux.sh
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Add Jetson cuda variants for arm This adds new variants for arm64 specific to Jetson platforms
2024-05-31 04:54:07 +00:00
FROM --platform=linux/arm64 nvidia/cuda:$CUDA_VERSION-devel-rockylinux8 AS cuda-build-server-arm64
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CMAKE_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} sh /rh_linux_deps.sh
Revert "Switch arm cuda base image to centos 7" This reverts commit 5dacc1ebe88fbd4fea355d4c664fdfb9547f360e.
2024-03-26 02:01:11 +00:00
ENV PATH /opt/rh/gcc-toolset-10/root/usr/bin:$PATH
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG CGO_CFLAGS
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
ENV GOARCH arm64
Add Jetson cuda variants for arm This adds new variants for arm64 specific to Jetson platforms
2024-05-31 04:54:07 +00:00
RUN OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_SKIP_CPU_GENERATE=1 bash gen_linux.sh
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK_6} AS cuda-build-jetpack6-arm64
ARG CMAKE_VERSION
RUN apt-get update && apt-get install -y git curl && \
curl -s -L https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}-linux-$(uname -m).tar.gz | tar -zx -C /usr --strip-components 1
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
ARG CGO_CFLAGS
ENV GOARCH arm64
ENV LIBRARY_PATH /usr/local/cuda/lib64/stubs
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 \
OLLAMA_SKIP_CPU_GENERATE=1 \
CUDA_VARIANT="_jetpack6" \
CUDA_DIST_DIR="/go/src/github.com/ollama/ollama/dist/linux-arm64/ollama_libs/cuda_jetpack6" \
CMAKE_CUDA_ARCHITECTURES="87" \
bash gen_linux.sh
FROM --platform=linux/arm64 nvcr.io/nvidia/l4t-jetpack:${JETPACK_5} AS cuda-build-jetpack5-arm64
ARG CMAKE_VERSION
RUN apt-get update && apt-get install -y git curl && \
curl -s -L https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}-linux-$(uname -m).tar.gz | tar -zx -C /usr --strip-components 1
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
ARG CGO_CFLAGS
ENV GOARCH arm64
ENV LIBRARY_PATH /usr/local/cuda/lib64/stubs
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 \
OLLAMA_SKIP_CPU_GENERATE=1 \
Add Jetson cuda variants for arm This adds new variants for arm64 specific to Jetson platforms
2024-05-31 04:54:07 +00:00
CUDA_VARIANT="_jetpack5" \
CUDA_DIST_DIR="/go/src/github.com/ollama/ollama/dist/linux-arm64/ollama_libs/cuda_jetpack5" \
CMAKE_CUDA_ARCHITECTURES="72;87" \
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
bash gen_linux.sh
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Revamp ROCm support This refines where we extract the LLM libraries to by adding a new OLLAMA_HOME env var, that defaults to `~/.ollama` The logic was already idempotenent, so this should speed up startups after the first time a new release is deployed. It also cleans up after itself. We now build only a single ROCm version (latest major) on both windows and linux. Given the large size of ROCms tensor files, we split the dependency out. It's bundled into the installer on windows, and a separate download on windows. The linux install script is now smart and detects the presence of AMD GPUs and looks to see if rocm v6 is already present, and if not, then downloads our dependency tar file. For Linux discovery, we now use sysfs and check each GPU against what ROCm supports so we can degrade to CPU gracefully instead of having llama.cpp+rocm assert/crash on us. For Windows, we now use go's windows dynamic library loading logic to access the amdhip64.dll APIs to query the GPU information.
2024-02-16 01:15:09 +00:00
FROM --platform=linux/amd64 rocm/dev-centos-7:${ROCM_VERSION}-complete AS rocm-build-amd64
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CMAKE_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} sh /rh_linux_deps.sh
ENV PATH /opt/rh/devtoolset-10/root/usr/bin:$PATH
ENV LIBRARY_PATH /opt/amdgpu/lib64
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG CGO_CFLAGS
ARG AMDGPU_TARGETS
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
ENV GOARCH amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_SKIP_CPU_GENERATE=1 bash gen_linux.sh
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
RUN mkdir -p ../../dist/linux-amd64/ollama_libs && \
(cd /opt/rocm/lib && tar cf - rocblas/library) | (cd ../../dist/linux-amd64/ollama_libs && tar xf - )
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
FROM --platform=linux/amd64 centos:7 AS cpu-builder-amd64
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CMAKE_VERSION
ARG GOLANG_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} GOLANG_VERSION=${GOLANG_VERSION} sh /rh_linux_deps.sh
ENV PATH /opt/rh/devtoolset-10/root/usr/bin:$PATH
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG OLLAMA_CUSTOM_CPU_DEFS
ARG CGO_CFLAGS
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
ENV GOARCH amd64
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
FROM --platform=linux/amd64 cpu-builder-amd64 AS static-build-amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_CPU_TARGET="static" bash gen_linux.sh
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
FROM --platform=linux/amd64 cpu-builder-amd64 AS cpu-build-amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_CPU_TARGET="cpu" bash gen_linux.sh
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
FROM --platform=linux/amd64 cpu-builder-amd64 AS cpu_avx-build-amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_CPU_TARGET="cpu_avx" bash gen_linux.sh
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
FROM --platform=linux/amd64 cpu-builder-amd64 AS cpu_avx2-build-amd64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_CPU_TARGET="cpu_avx2" bash gen_linux.sh
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Switch amd container image base to rocky 8 The centos 7 arm mirrors have disappeared due to the EOL 2 days ago, and the vault sed workaround which works for x86 doesn't work for arm.
2024-07-02 17:23:05 +00:00
FROM --platform=linux/arm64 rockylinux:8 AS cpu-builder-arm64
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ARG CMAKE_VERSION
ARG GOLANG_VERSION
COPY ./scripts/rh_linux_deps.sh /
RUN CMAKE_VERSION=${CMAKE_VERSION} GOLANG_VERSION=${GOLANG_VERSION} sh /rh_linux_deps.sh
Switch amd container image base to rocky 8 The centos 7 arm mirrors have disappeared due to the EOL 2 days ago, and the vault sed workaround which works for x86 doesn't work for arm.
2024-07-02 17:23:05 +00:00
ENV PATH /opt/rh/gcc-toolset-10/root/usr/bin:$PATH
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
COPY --from=llm-code / /go/src/github.com/ollama/ollama/
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG OLLAMA_CUSTOM_CPU_DEFS
ARG CGO_CFLAGS
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
ENV GOARCH arm64
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
WORKDIR /go/src/github.com/ollama/ollama/llm/generate
FROM --platform=linux/arm64 cpu-builder-arm64 AS static-build-arm64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_CPU_TARGET="static" bash gen_linux.sh
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
FROM --platform=linux/arm64 cpu-builder-arm64 AS cpu-build-arm64
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
OLLAMA_SKIP_STATIC_GENERATE=1 OLLAMA_CPU_TARGET="cpu" bash gen_linux.sh
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
# Intermediate stage used for ./scripts/build_linux.sh
FROM --platform=linux/amd64 cpu-build-amd64 AS build-amd64
ENV CGO_ENABLED 1
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
WORKDIR /go/src/github.com/ollama/ollama
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
COPY . .
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
COPY --from=static-build-amd64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
COPY --from=cpu_avx-build-amd64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
COPY --from=cpu_avx2-build-amd64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
COPY --from=cuda-build-amd64 /go/src/github.com/ollama/ollama/dist/ dist/
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
COPY --from=cuda-build-amd64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
COPY --from=rocm-build-amd64 /go/src/github.com/ollama/ollama/dist/ dist/
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
COPY --from=rocm-build-amd64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG GOFLAGS
ARG CGO_CFLAGS
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
go build -trimpath -o dist/linux-amd64/ollama .
update dockerfile
2023-09-06 21:14:20 +00:00
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
# Intermediate stage used for ./scripts/build_linux.sh
FROM --platform=linux/arm64 cpu-build-arm64 AS build-arm64
ENV CGO_ENABLED 1
ARG GOLANG_VERSION
change `github.com/jmorganca/ollama` to `github.com/ollama/ollama` (#3347)
2024-03-26 20:04:17 +00:00
WORKDIR /go/src/github.com/ollama/ollama
update `Dockerfile`
2023-07-03 22:56:17 +00:00
COPY . .
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 17:24:13 +00:00
COPY --from=static-build-arm64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
Add Jetson cuda variants for arm This adds new variants for arm64 specific to Jetson platforms
2024-05-31 04:54:07 +00:00
COPY --from=cuda-build-server-arm64 /go/src/github.com/ollama/ollama/dist/ dist/
COPY --from=cuda-build-server-arm64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
## arm binary += 381M
COPY --from=cuda-build-jetpack6-arm64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
COPY --from=cuda-build-jetpack6-arm64 /go/src/github.com/ollama/ollama/dist/ dist/
## arm binary += 330M
COPY --from=cuda-build-jetpack5-arm64 /go/src/github.com/ollama/ollama/llm/build/linux/ llm/build/linux/
COPY --from=cuda-build-jetpack5-arm64 /go/src/github.com/ollama/ollama/dist/ dist/
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 20:57:13 +00:00
ARG GOFLAGS
ARG CGO_CFLAGS
Wire up ccache and pigz in the docker based build This should help speed things up a little
2024-08-09 14:21:40 +00:00
RUN --mount=type=cache,target=/root/.ccache \
go build -trimpath -o dist/linux-arm64/ollama .
update `Dockerfile`
2023-07-03 22:56:17 +00:00
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
# Runtime stages
Switch back to ubuntu base The size increase for rocm support in the standard image is problematic We'll revisit multiple tags for rocm support in a follow up PR.
2024-01-26 00:46:01 +00:00
FROM --platform=linux/amd64 ubuntu:22.04 as runtime-amd64
RUN apt-get update && apt-get install -y ca-certificates
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
COPY --from=build-amd64 /go/src/github.com/ollama/ollama/dist/linux-amd64/ollama /bin/ollama
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
FROM --platform=linux/arm64 ubuntu:22.04 as runtime-arm64
replace dockerfile
2023-09-22 18:57:31 +00:00
RUN apt-get update && apt-get install -y ca-certificates
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
COPY --from=build-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64/ollama /bin/ollama
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
Add back ROCm container support This adds ROCm support back as a discrete image.
2024-01-26 00:58:05 +00:00
# Radeon images are much larger so we keep it distinct from the CPU/CUDA image
Revamp ROCm support This refines where we extract the LLM libraries to by adding a new OLLAMA_HOME env var, that defaults to `~/.ollama` The logic was already idempotenent, so this should speed up startups after the first time a new release is deployed. It also cleans up after itself. We now build only a single ROCm version (latest major) on both windows and linux. Given the large size of ROCms tensor files, we split the dependency out. It's bundled into the installer on windows, and a separate download on windows. The linux install script is now smart and detects the presence of AMD GPUs and looks to see if rocm v6 is already present, and if not, then downloads our dependency tar file. For Linux discovery, we now use sysfs and check each GPU against what ROCm supports so we can degrade to CPU gracefully instead of having llama.cpp+rocm assert/crash on us. For Windows, we now use go's windows dynamic library loading logic to access the amdhip64.dll APIs to query the GPU information.
2024-02-16 01:15:09 +00:00
FROM --platform=linux/amd64 rocm/dev-centos-7:${ROCM_VERSION}-complete as runtime-rocm
Add back ROCm container support This adds ROCm support back as a discrete image.
2024-01-26 00:58:05 +00:00
RUN update-pciids
Refactor linux packaging This adjusts linux to follow a similar model to windows with a discrete archive (zip/tgz) to cary the primary executable, and dependent libraries. Runners are still carried as payloads inside the main binary Darwin retain the payload model where the go binary is fully self contained.
2024-07-08 19:50:11 +00:00
COPY --from=build-amd64 /go/src/github.com/ollama/ollama/dist/linux-amd64/ollama /bin/ollama
Add back ROCm container support This adds ROCm support back as a discrete image.
2024-01-26 00:58:05 +00:00
EXPOSE 11434
ENV OLLAMA_HOST 0.0.0.0
ENTRYPOINT ["/bin/ollama"]
CMD ["serve"]
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
FROM runtime-$TARGETARCH
docker: fix volume permission errors
2023-09-30 19:32:06 +00:00
EXPOSE 11434
ENV OLLAMA_HOST 0.0.0.0
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 19:37:11 +00:00
ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Add back ROCm container support This adds ROCm support back as a discrete image.
2024-01-26 00:58:05 +00:00
ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
docker: set PATH, LD_LIBRARY_PATH, and capabilities (#1336) * docker: set PATH, LD_LIBRARY_PATH, and capabilities * example: update k8s gpu manifest
2023-12-01 05:16:56 +00:00
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
Add env var so podman will map cuda GPUs Without this env var, podman's GPU logic doesn't map the GPU through
2024-02-29 16:43:08 +00:00
ENV NVIDIA_VISIBLE_DEVICES=all
docker: set PATH, LD_LIBRARY_PATH, and capabilities (#1336) * docker: set PATH, LD_LIBRARY_PATH, and capabilities * example: update k8s gpu manifest
2023-12-01 05:16:56 +00:00
update `Dockerfile`
2023-07-03 22:56:17 +00:00
ENTRYPOINT ["/bin/ollama"]
fix dockerfile
2023-07-06 20:34:14 +00:00
CMD ["serve"]
Reference in a new issue Copy permalink