ollama/Dockerfile

FROM intel/oneapi-basekit:2024.2.0-devel-rockylinux9 as build

RUN wget $(echo "https://pkgs.dyn.su/el9/base/x86_64/raven-release.el9.noarch.rpm" | sed "s/el9/el$(rpm -q --queryformat '%{RELEASE}' rpm | grep -oP 'el\K[0-9]+')/g") && \
    rpm -ivh raven-release*.rpm && \
    rm -rf raven-release*.rpm && \
    dnf update -y && \
    dnf -y --enablerepo=raven-extras install golang

WORKDIR /app

ADD . .

RUN OLLAMA_CUSTOM_CPU_DEFS="-DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=Intel10_64lp -DLLAMA_NATIVE=ON" go generate ./... && \
    go build -ldflags="-s -w"

FROM oraclelinux:9

RUN echo -e "[oneAPI]\n\
name=Intel® oneAPI repository\n\
baseurl=https://yum.repos.intel.com/oneapi\n\
enabled=1\n\
gpgcheck=1\n\
repo_gpgcheck=1\n\
gpgkey=https://yum.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB" > /tmp/oneAPI.repo

RUN mv /tmp/oneAPI.repo /etc/yum.repos.d/oneAPI.repo && \
    dnf update && \
    dnf install intel-oneapi-mkl -y && \
    mkdir /ollama

COPY --from=build /app/ollama /ollama

ADD entrypoint.sh /

ENV MODEL_NAME="llama"
ENV OLLAMA_HOST="0.0.0.0:8080"

EXPOSE 8080

ENTRYPOINT ["/entrypoint.sh"]

CMD ["/ollama/ollama", "run", "model"]
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`FROM intel/oneapi-basekit:2024.2.0-devel-rockylinux9 as build`
replace dockerfile 2023-09-22 18:57:31 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`RUN wget $(echo "https://pkgs.dyn.su/el9/base/x86_64/raven-release.el9.noarch.rpm" \| sed "s/el9/el$(rpm -q --queryformat '%{RELEASE}' rpm \| grep -oP 'el\K[0-9]+')/g") && \`
			`rpm -ivh raven-release*.rpm && \`
			`rm -rf raven-release*.rpm && \`
			`dnf update -y && \`
			`dnf -y --enablerepo=raven-extras install golang`
update cuda docker image 2023-09-08 20:41:49 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`WORKDIR /app`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`ADD . .`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`RUN OLLAMA_CUSTOM_CPU_DEFS="-DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=Intel10_64lp -DLLAMA_NATIVE=ON" go generate ./... && \`
			`go build -ldflags="-s -w"`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`FROM oraclelinux:9`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`RUN echo -e "[oneAPI]\n\`
			`name=Intel® oneAPI repository\n\`
			`baseurl=https://yum.repos.intel.com/oneapi\n\`
			`enabled=1\n\`
			`gpgcheck=1\n\`
			`repo_gpgcheck=1\n\`
			`gpgkey=https://yum.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB" > /tmp/oneAPI.repo`
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set. 2024-01-21 20:57:13 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`RUN mv /tmp/oneAPI.repo /etc/yum.repos.d/oneAPI.repo && \`
			`dnf update && \`
			`dnf install intel-oneapi-mkl -y && \`
			`mkdir /ollama`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`COPY --from=build /app/ollama /ollama`
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently. 2024-03-14 17:24:13 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`ADD entrypoint.sh /`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`ENV MODEL_NAME="llama"`
			`ENV OLLAMA_HOST="0.0.0.0:8080"`
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently. 2024-03-14 17:24:13 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`EXPOSE 8080`
update dockerfile 2023-09-06 21:14:20 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`ENTRYPOINT ["/entrypoint.sh"]`
update `Dockerfile` 2023-07-03 22:56:17 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`CMD ["/ollama/ollama", "run", "model"]`