ollama/Dockerfile

FROM golang:1.22-alpine as build

RUN apk add cmake build-base bash git

WORKDIR /app

ADD . .

ARG GIN_MODE=release

RUN OLLAMA_CUSTOM_CPU_DEFS="-DLLAMA_NATIVE=ON" go generate ./... && \
    go build -ldflags="-s -w"

FROM alpine:latest

RUN apk add libstdc++ libgomp supervisor

WORKDIR /ollama

COPY --from=build /app/ollama /ollama
COPY --from=build /app/serve.sh /ollama/serve.sh
COPY --from=build /app/run_model.sh /ollama/run_model.sh
COPY --from=build /app/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
COPY --from=build /app/llm/build/linux/x86_64/cpu/src/libllama.so /usr/lib
COPY --from=build /app/llm/build/linux/x86_64/cpu/ggml/src/libggml.so /usr/lib

ENV MODEL_NAME="llama"
ENV OLLAMA_HOST="0.0.0.0:8080"

EXPOSE 8080

CMD ["supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-07 16:19:35 +00:00			`FROM golang:1.22-alpine as build`
replace dockerfile 2023-09-22 18:57:31 +00:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-07 16:19:35 +00:00			`RUN apk add cmake build-base bash git`
update cuda docker image 2023-09-08 20:41:49 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`WORKDIR /app`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`ADD . .`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-07 16:19:35 +00:00			`ARG GIN_MODE=release`

			`RUN OLLAMA_CUSTOM_CPU_DEFS="-DLLAMA_NATIVE=ON" go generate ./... && \`
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`go build -ldflags="-s -w"`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-07 16:19:35 +00:00			`FROM alpine:latest`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-07 16:19:35 +00:00			`RUN apk add libstdc++ libgomp supervisor`
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set. 2024-01-21 20:57:13 +00:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-07 16:19:35 +00:00			`WORKDIR /ollama`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`COPY --from=build /app/ollama /ollama`
Make run model a oneshot service Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 12:44:10 +00:00			`COPY --from=build /app/serve.sh /ollama/serve.sh`
			`COPY --from=build /app/run_model.sh /ollama/run_model.sh`
			`COPY --from=build /app/supervisord.conf /etc/supervisor/conf.d/supervisord.conf`
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-07 16:19:35 +00:00			`COPY --from=build /app/llm/build/linux/x86_64/cpu/src/libllama.so /usr/lib`
			`COPY --from=build /app/llm/build/linux/x86_64/cpu/ggml/src/libggml.so /usr/lib`
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat. 2024-01-21 19:37:11 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`ENV MODEL_NAME="llama"`
			`ENV OLLAMA_HOST="0.0.0.0:8080"`
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently. 2024-03-14 17:24:13 +00:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-01 11:14:31 +00:00			`EXPOSE 8080`
update dockerfile 2023-09-06 21:14:20 +00:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me> 2024-07-07 16:19:35 +00:00			`CMD ["supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]`