Remove Modelfile parameters that are decided at runtime
12 KiB
Ollama Model File
Note:
Modelfile
syntax is in development
A model file is the blueprint to create and share models with Ollama.
Table of Contents
Format
The format of the Modelfile
:
# comment
INSTRUCTION arguments
Instruction | Description |
---|---|
FROM (required) |
Defines the base model to use. |
PARAMETER |
Sets the parameters for how Ollama will run the model. |
TEMPLATE |
The full prompt template to be sent to the model. |
SYSTEM |
Specifies the system message that will be set in the template. |
ADAPTER |
Defines the (Q)LoRA adapters to apply to the model. |
LICENSE |
Specifies the legal license. |
MESSAGE |
Specify message history. |
Examples
Basic Modelfile
An example of a Modelfile
creating a mario blueprint:
FROM llama2
# sets the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1
# sets the context window size to 4096, this controls how many tokens the LLM can use as context to generate the next token
PARAMETER num_ctx 4096
# sets a custom system message to specify the behavior of the chat assistant
SYSTEM You are Mario from super mario bros, acting as an assistant.
To use this:
- Save it as a file (e.g.
Modelfile
) ollama create choose-a-model-name -f <location of the file e.g. ./Modelfile>'
ollama run choose-a-model-name
- Start using the model!
More examples are available in the examples directory.
Modelfile
s in ollama.com/library
There are two ways to view Modelfile
s underlying the models in ollama.com/library:
-
Option 1: view a details page from a model's tags page:
- Go to a particular model's tags (e.g. https://ollama.com/library/llama2/tags)
- Click on a tag (e.g. https://ollama.com/library/llama2:13b)
- Scroll down to "Layers"
- Note: if the
FROM
instruction is not present, it means the model was created from a local file
- Note: if the
-
Option 2: use
ollama show
to print theModelfile
for any local models like so:> ollama show --modelfile llama2:13b # Modelfile generated by "ollama show" # To build a new Modelfile based on this one, replace the FROM line with: # FROM llama2:13b FROM /root/.ollama/models/blobs/sha256:123abc TEMPLATE """[INST] {{ if .System }}<<SYS>>{{ .System }}<</SYS>> {{ end }}{{ .Prompt }} [/INST] """ SYSTEM """""" PARAMETER stop [INST] PARAMETER stop [/INST] PARAMETER stop <<SYS>> PARAMETER stop <</SYS>>
Instructions
FROM (Required)
The FROM
instruction defines the base model to use when creating a model.
FROM <model name>:<tag>
Build from llama2
FROM llama2
A list of available base models: https://github.com/ollama/ollama#model-library
Build from a bin
file
FROM ./ollama-model.bin
This bin file location should be specified as an absolute path or relative to the Modelfile
location.
PARAMETER
The PARAMETER
instruction defines a parameter that can be set when the model is run.
PARAMETER <parameter> <parametervalue>
Valid Parameters and Values
Parameter | Description | Value Type | Example Usage |
---|---|---|---|
mirostat | Enable Mirostat sampling for controlling perplexity. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0) | int | mirostat 0 |
mirostat_eta | Influences how quickly the algorithm responds to feedback from the generated text. A lower learning rate will result in slower adjustments, while a higher learning rate will make the algorithm more responsive. (Default: 0.1) | float | mirostat_eta 0.1 |
mirostat_tau | Controls the balance between coherence and diversity of the output. A lower value will result in more focused and coherent text. (Default: 5.0) | float | mirostat_tau 5.0 |
num_ctx | Sets the size of the context window used to generate the next token. (Default: 2048) | int | num_ctx 4096 |
repeat_last_n | Sets how far back for the model to look back to prevent repetition. (Default: 64, 0 = disabled, -1 = num_ctx) | int | repeat_last_n 64 |
repeat_penalty | Sets how strongly to penalize repetitions. A higher value (e.g., 1.5) will penalize repetitions more strongly, while a lower value (e.g., 0.9) will be more lenient. (Default: 1.1) | float | repeat_penalty 1.1 |
temperature | The temperature of the model. Increasing the temperature will make the model answer more creatively. (Default: 0.8) | float | temperature 0.7 |
seed | Sets the random number seed to use for generation. Setting this to a specific number will make the model generate the same text for the same prompt. (Default: 0) | int | seed 42 |
stop | Sets the stop sequences to use. When this pattern is encountered the LLM will stop generating text and return. Multiple stop patterns may be set by specifying multiple separate stop parameters in a modelfile. |
string | stop "AI assistant:" |
tfs_z | Tail free sampling is used to reduce the impact of less probable tokens from the output. A higher value (e.g., 2.0) will reduce the impact more, while a value of 1.0 disables this setting. (default: 1) | float | tfs_z 1 |
num_predict | Maximum number of tokens to predict when generating text. (Default: 128, -1 = infinite generation, -2 = fill context) | int | num_predict 42 |
top_k | Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative. (Default: 40) | int | top_k 40 |
top_p | Works together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text. (Default: 0.9) | float | top_p 0.9 |
TEMPLATE
TEMPLATE
of the full prompt template to be passed into the model. It may include (optionally) a system message, a user's message and the response from the model. Note: syntax may be model specific. Templates use Go template syntax.
Template Variables
Variable | Description |
---|---|
{{ .System }} |
The system message used to specify custom behavior. |
{{ .Prompt }} |
The user prompt message. |
{{ .Response }} |
The response from the model. When generating a response, text after this variable is omitted. |
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
"""
SYSTEM
The SYSTEM
instruction specifies the system message to be used in the template, if applicable.
SYSTEM """<system message>"""
ADAPTER
The ADAPTER
instruction is an optional instruction that specifies any LoRA adapter that should apply to the base model. The value of this instruction should be an absolute path or a path relative to the Modelfile and the file must be in a GGML file format. The adapter should be tuned from the base model otherwise the behaviour is undefined.
ADAPTER ./ollama-lora.bin
LICENSE
The LICENSE
instruction allows you to specify the legal license under which the model used with this Modelfile is shared or distributed.
LICENSE """
<license text>
"""
MESSAGE
The MESSAGE
instruction allows you to specify a message history for the model to use when responding. Use multiple iterations of the MESSAGE command to build up a conversation which will guide the model to answer in a similar way.
MESSAGE <role> <message>
Valid roles
Role | Description |
---|---|
system | Alternate way of providing the SYSTEM message for the model. |
user | An example message of what the user could have asked. |
assistant | An example message of how the model should respond. |
Example conversation
MESSAGE user Is Toronto in Canada?
MESSAGE assistant yes
MESSAGE user Is Sacramento in Canada?
MESSAGE assistant no
MESSAGE user Is Ontario in Canada?
MESSAGE assistant yes
Notes
- the
Modelfile
is not case sensitive. In the examples, uppercase instructions are used to make it easier to distinguish it from arguments. - Instructions can be in any order. In the examples, the
FROM
instruction is first to keep it easily readable.