Add chat template to tokenizer config

#6
by saattrupdan - opened

When working with the tokenizer in transformers, the chat template is not included and tokenizer.apply_chat_template for instance doesn't work.

This adds the chat template to the config to remedy this. It might also work if the current chat_template.json is replaced with a chat_template.jinja file, only containing the chat template string, but I'm not completely sure about that.

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment