Integration of Jinja2 Templating (#875 )

* feat: Add support for jinja templating

Signed-off-by: teleprint-me <77757836+teleprint-me@users.noreply.github.com>

* fix: Refactor chat formatter and update interface for jinja templates

- Simplify the `llama2_template` in `llama_jinja_format.py` by removing unnecessary line breaks for readability without affecting functionality.
- Update `ChatFormatterInterface` constructor to accept a more generic `Optional[object]` type for the template parameter, enhancing flexibility.
- Introduce a `template` property to `ChatFormatterInterface` for standardized access to the template string.
- Replace `MetaSingleton` metaclass with `Singleton` for the `ChatFormatterFactory` to streamline the singleton implementation.

These changes enhance code readability, maintain usability, and ensure consistency in the chat formatter's design pattern usage.

* Add outline for Jinja2 templating integration documentation

Signed-off-by: teleprint-me <77757836+teleprint-me@users.noreply.github.com>

* Add jinja2 as a dependency with version range for Hugging Face transformers compatibility

Signed-off-by: teleprint-me <77757836+teleprint-me@users.noreply.github.com>

* Update jinja2 version constraint for mkdocs-material compatibility

Signed-off-by: teleprint-me <77757836+teleprint-me@users.noreply.github.com>

* Fix attribute name in AutoChatFormatter

- Changed attribute name from `self._renderer` to `self._environment`

---------

Signed-off-by: teleprint-me <77757836+teleprint-me@users.noreply.github.com>

2024-01-17 09:47:52 -05:00

2 KiB

Raw Blame History

Templates

This document provides a comprehensive guide to the integration of Jinja2 templating into the llama-cpp-python project, with a focus on enhancing the chat functionality of the llama-2 model.

Introduction

Brief explanation of the llama-cpp-python project's need for a templating system.
Overview of the llama-2 model's interaction with templating.

Jinja2 Dependency Integration

Rationale for choosing Jinja2 as the templating engine.
- Compatibility with Hugging Face's transformers.
- Desire for advanced templating features and simplicity.
Detailed steps for adding jinja2 to pyproject.toml for dependency management.

Template Management Refactor

Summary of the refactor and the motivation behind it.
Description of the new chat handler selection logic:
1. Preference for a user-specified chat_handler.
2. Fallback to a user-specified chat_format.
3. Defaulting to a chat format from a .gguf file if available.
4. Utilizing the llama2 default chat format as the final fallback.
Ensuring backward compatibility throughout the refactor.

Implementation Details

In-depth look at the new AutoChatFormatter class.
Example code snippets showing how to utilize the Jinja2 environment and templates.
Guidance on how to provide custom templates or use defaults.

Testing and Validation

Outline of the testing strategy to ensure seamless integration.
Steps for validating backward compatibility with existing implementations.

Benefits and Impact

Analysis of the expected benefits, including consistency, performance gains, and improved developer experience.
Discussion of the potential impact on current users and contributors.

Future Work

Exploration of how templating can evolve within the project.
Consideration of additional features or optimizations for the templating engine.
Mechanisms for community feedback on the templating system.

Conclusion

Final thoughts on the integration of Jinja2 templating.
Call to action for community involvement and feedback.

2 KiB Raw Blame History