baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

docs: Add n_ctx not for multimodal models

Browse source
This commit is contained in:
Andrei Betlen 2023-11-22 21:07:00 -05:00
parent 36679a58ef
commit abb1976ad7
1 changed files with 5 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
README.md
View file

@ -233,7 +233,11 @@ Then you'll need to use a custom chat handler to load the clip model and process
>>> from llama_cpp import Llama >>> from llama_cpp import Llama
>>> from llama_cpp.llama_chat_format import Llava15ChatHandler >>> from llama_cpp.llama_chat_format import Llava15ChatHandler
>>> chat_handler = Llava15ChatHandler(clip_model_path="path/to/llava/mmproj.bin") >>> chat_handler = Llava15ChatHandler(clip_model_path="path/to/llava/mmproj.bin")
>>> llm = Llama(model_path="./path/to/llava/llama-model.gguf", chat_handler=chat_handler) >>> llm = Llama(
model_path="./path/to/llava/llama-model.gguf",
chat_handler=chat_handler,
n_ctx=2048 # n_ctx should be increased to accomodate the image embedding
)
>>> llm.create_chat_completion( >>> llm.create_chat_completion(
messages = [ messages = [
{"role": "system", "content": "You are an assistant who perfectly describes images."}, {"role": "system", "content": "You are an assistant who perfectly describes images."},