baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

set llama_max_devices using library function

Browse source
This commit is contained in:
Andrei Betlen 2023-12-22 15:19:28 -05:00
parent d9a1d90fd7
commit 2b0d3f36fa
1 changed files with 1 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
llama_cpp/llama_cpp.py
View file

@ -90,9 +90,7 @@ c_size_t_p = POINTER(c_size_t)
# llama.h bindings
GGML_USE_CUBLAS = hasattr(_lib, "ggml_init_cublas")
GGML_CUDA_MAX_DEVICES = 16
LLAMA_MAX_DEVICES = GGML_CUDA_MAX_DEVICES if GGML_USE_CUBLAS else 1
LLAMA_MAX_DEVICES = _lib.llama_max_devices()
# define LLAMA_DEFAULT_SEED 0xFFFFFFFF
LLAMA_DEFAULT_SEED = 0xFFFFFFFF