Commit graph

1 commit

Author SHA1 Message Date
Antoine Lizee
4d4e0f11e2 fix: tokenization of special characters: (#850)
It should behave like llama.cpp, where most out of the box usages
treat special characters accordingly
2023-11-02 14:28:14 -04:00