From 6ae33d8141585583e840a45bb624ddd81ccfdf88 Mon Sep 17 00:00:00 2001 From: Aaron Coffey Date: Thu, 28 Sep 2023 07:21:21 -0700 Subject: [PATCH] Update modelfile.md to reflect the usage of num_gpu. (#629) --- docs/modelfile.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/modelfile.md b/docs/modelfile.md index 1609ed3a..ade9c7c0 100644 --- a/docs/modelfile.md +++ b/docs/modelfile.md @@ -119,7 +119,7 @@ PARAMETER | mirostat_tau | Controls the balance between coherence and diversity of the output. A lower value will result in more focused and coherent text. (Default: 5.0) | float | mirostat_tau 5.0 | | num_ctx | Sets the size of the context window used to generate the next token. (Default: 2048) | int | num_ctx 4096 | | num_gqa | The number of GQA groups in the transformer layer. Required for some models, for example it is 8 for llama2:70b | int | num_gqa 1 | -| num_gpu | The number of GPUs to use. On macOS it defaults to 1 to enable metal support, 0 to disable. | int | num_gpu 1 | +| num_gpu | The number of layers to send to the GPU(s). On macOS it defaults to 1 to enable metal support, 0 to disable. | int | num_gpu 50 | | num_thread | Sets the number of threads to use during computation. By default, Ollama will detect this for optimal performance. It is recommended to set this value to the number of physical CPU cores your system has (as opposed to the logical number of cores). | int | num_thread 8 | | repeat_last_n | Sets how far back for the model to look back to prevent repetition. (Default: 64, 0 = disabled, -1 = num_ctx) | int | repeat_last_n 64 | | repeat_penalty | Sets how strongly to penalize repetitions. A higher value (e.g., 1.5) will penalize repetitions more strongly, while a lower value (e.g., 0.9) will be more lenient. (Default: 1.1) | float | repeat_penalty 1.1 |