baalajimaestro/ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
ollama/Dockerfile

33 lines
814 B
Text
Raw Normal View History

Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-07 21:49:35 +05:30
FROM golang:1.22-alpine as build
replace dockerfile
2023-09-22 11:57:31 -07:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-07 21:49:35 +05:30
RUN apk add cmake build-base bash git
update cuda docker image
2023-09-08 13:41:49 -07:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-01 16:44:31 +05:30
WORKDIR /app
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 11:37:11 -08:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-01 16:44:31 +05:30
ADD . .
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 11:37:11 -08:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-07 21:49:35 +05:30
ARG GIN_MODE=release
RUN OLLAMA_CUSTOM_CPU_DEFS="-DLLAMA_NATIVE=ON" go generate ./... && \
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-01 16:44:31 +05:30
go build -ldflags="-s -w"
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 11:37:11 -08:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-07 21:49:35 +05:30
FROM alpine:latest
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 11:37:11 -08:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-07 21:49:35 +05:30
RUN apk add libstdc++ libgomp supervisor
Make CPU builds parallel and customizable AMD GPUs The linux build now support parallel CPU builds to speed things up. This also exposes AMD GPU targets as an optional setting for advaced users who want to alter our default set.
2024-01-21 12:57:13 -08:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-07 21:49:35 +05:30
WORKDIR /ollama
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 11:37:11 -08:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-01 16:44:31 +05:30
COPY --from=build /app/ollama /ollama
Make run model a oneshot service Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-01 18:14:10 +05:30
COPY --from=build /app/serve.sh /ollama/serve.sh
COPY --from=build /app/run_model.sh /ollama/run_model.sh
COPY --from=build /app/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-07 21:49:35 +05:30
COPY --from=build /app/llm/build/linux/x86_64/cpu/src/libllama.so /usr/lib
COPY --from=build /app/llm/build/linux/x86_64/cpu/ggml/src/libggml.so /usr/lib
Combine the 2 Dockerfiles and add ROCm This renames Dockerfile.build to Dockerfile, and adds some new stages to support 2 modes of building - the build_linux.sh script uses intermediate stages to extract the artifacts for ./dist, and the default build generates a container image usable by both cuda and rocm cards. This required transitioniing the x86 base to the rocm image to avoid layer bloat.
2024-01-21 11:37:11 -08:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-01 16:44:31 +05:30
ENV MODEL_NAME="llama"
ENV OLLAMA_HOST="0.0.0.0:8080"
Switch back to subprocessing for llama.cpp This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
2024-03-14 10:24:13 -07:00
Add building on oneapi Also handle model names easily for docker Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-01 16:44:31 +05:30
EXPOSE 8080
update dockerfile
2023-09-06 14:14:20 -07:00
Use alpine and remove blas Signed-off-by: baalajimaestro <me@baalajimaestro.me>
2024-07-07 21:49:35 +05:30
CMD ["supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]
Reference in a new issue Copy permalink