ollama/llm
Daniel Hiltgen cd5c8f6471
Optimize container images for startup (#6547)
* Optimize container images for startup

This change adjusts how to handle runner payloads to support
container builds where we keep them extracted in the filesystem.
This makes it easier to optimize the cpu/cuda vs cpu/rocm images for
size, and should result in faster startup times for container images.

* Refactor payload logic and add buildx support for faster builds

* Move payloads around

* Review comments

* Converge to buildx based helper scripts

* Use docker buildx action for release
2024-09-12 12:10:30 -07:00
..
ext_server runner: Flush pending responses before returning 2024-09-11 16:39:32 -07:00
generate Optimize container images for startup (#6547) 2024-09-12 12:10:30 -07:00
llama.cpp@8962422b1c llm: update llama.cpp commit to 8962422 (#6618) 2024-09-03 21:12:39 -04:00
patches llm: update llama.cpp commit to 8962422 (#6618) 2024-09-03 21:12:39 -04:00
filetype.go Add support for IQ1_S, IQ3_S, IQ2_S, IQ4_XS. IQ4_NL (#4322) 2024-05-23 13:21:49 -07:00
ggla.go update convert test to check result data 2024-07-31 10:59:38 -07:00
ggml.go Merge pull request #6260 from ollama/mxyng/mem 2024-09-05 13:22:08 -07:00
ggml_test.go llm: speed up gguf decoding by a lot (#5246) 2024-06-24 21:47:52 -07:00
gguf.go add conversion for microsoft phi 3 mini/medium 4k, 128 2024-08-12 15:13:29 -07:00
llm.go lint 2024-08-01 17:06:06 -07:00
llm_darwin.go Optimize container images for startup (#6547) 2024-09-12 12:10:30 -07:00
llm_linux.go Optimize container images for startup (#6547) 2024-09-12 12:10:30 -07:00
llm_windows.go Optimize container images for startup (#6547) 2024-09-12 12:10:30 -07:00
memory.go Improve logging on GPU too small (#6666) 2024-09-06 08:29:36 -07:00
memory_test.go llama3.1 2024-08-21 11:49:31 -07:00
server.go Optimize container images for startup (#6547) 2024-09-12 12:10:30 -07:00
status.go Catch one more error log 2024-08-05 09:28:07 -07:00