baalajimaestro/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

docs: Update README.md

Browse source
This commit is contained in:
Andrei Betlen 2024-04-30 10:11:38 -04:00
parent b14dd98922
commit 26478ab293
1 changed files with 10 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
README.md
View file

@ -529,19 +529,22 @@ Then you'll need to use a custom chat handler to load the clip model and process
You can also pull the model from the Hugging Face Hub using the `from_pretrained` method. You can also pull the model from the Hugging Face Hub using the `from_pretrained` method.
```python ```python
>>> from llama_cpp import Llama from llama_cpp import Llama
>>> from llama_cpp.llama_chat_format import MoondreamChatHandler from llama_cpp.llama_chat_format import MoondreamChatHandler
>>> chat_handler = MoondreamChatHandler.from_pretrained(
chat_handler = MoondreamChatHandler.from_pretrained(
repo_id="vikhyatk/moondream2", repo_id="vikhyatk/moondream2",
filename="*mmproj*", filename="*mmproj*",
) )
>>> llm = Llama.from_pretrained(
repo_id="vikhyatk/moondream2" llm = Llama.from_pretrained(
repo_id="vikhyatk/moondream2",
filename="*text-model*", filename="*text-model*",
chat_handler=chat_handler, chat_handler=chat_handler,
n_ctx=2048, # n_ctx should be increased to accomodate the image embedding n_ctx=2048, # n_ctx should be increased to accomodate the image embedding
) )
>>> llm.create_chat_completion(
respoonse = llm.create_chat_completion(
messages = [ messages = [
{ {
"role": "user", "role": "user",
@ -553,6 +556,7 @@ You can also pull the model from the Hugging Face Hub using the `from_pretrained
} }
] ]
) )
print(response["choices"][0]["text"])
``` ```
**Note**: Multi-modal models also support tool calling and JSON mode. **Note**: Multi-modal models also support tool calling and JSON mode.