Why chat needs a template
A causal language model reads a sequence of tokens, not a user-interface array of colored message bubbles. The template bridges those representations. It might put a special token before each role, add end-of-turn markers, and append a marker telling the model that the assistant should speak next.
Two models derived from the same base model can be fine-tuned with different control tokens. A prompt format that works for one can reduce quality or make another continue the wrong speaker.
How it is applied
Libraries and inference servers commonly store the template with tokenizer configuration and apply it before tokenization or generation. The input remains a list of role-and-content messages for the application, while the rendered result contains literal text and special tokens. Some templates support tools, documents, or multimodal content; others recognize only a basic user/assistant exchange.
A generation prompt is often appended to mark the start of the assistant turn. During training, a related continuation flag may instead preserve an unfinished final message. These details are implementation-specific, which is why copying a template from an unrelated model is unreliable.
Why it matters for local adult-AI chat
Local interfaces such as SillyTavern, KoboldCpp, and LM Studio may expose a template choice or infer one from model metadata. The template affects whether a system prompt, character card, and chat history reach the model in the form it was trained to follow. It does not remove the model's policy behavior or create capabilities absent from its training.
Chat template versus prompt template
Prompt template is a broad label for any reusable prompt layout. A chat template specifically serializes a conversation according to a model's role and control-token convention. The visible character prompt can be one input to that process, but it is not a substitute for the low-level chat format.
Common failure modes
- Wrong role markers: the model may echo instructions, speak as the user, or stop at the wrong place.
- Duplicated special tokens: adding tokens both in the template and tokenizer can degrade results.
- Missing metadata: an application may need a manual choice when a model repository does not publish its expected format.