The model answers from the current context
On each turn, the model receives a constructed prompt: system instructions, character definition, user persona, scenario, selected memories or lore, recent messages, and space for a reply. The context window limits how many tokens fit. Text outside that constructed prompt is not directly visible to the model on that turn.
This is why a chat log stored by an application is not automatically active memory. The application must keep, summarize, or retrieve relevant information and insert it into the prompt.
Separate the main memory layers
- Permanent prompt material: character identity, scenario, core boundaries, and system instructions sent repeatedly. Reliable but consumes context every turn.
- Recent conversation: the newest messages kept verbatim until older turns are trimmed.
- Summary: a compressed account of earlier events. Efficient, but detail and exact wording can be lost.
- Retrieved memory: selected stored facts inserted when relevant, often using retrieval-augmented generation-like methods.
- Lore or world info: entries activated by keywords or rules. SillyTavern exposes budgets, scan depth, and context-specific lorebooks.
A local KV cache speeds inference over active context; it is not a durable autobiographical memory store.
Budget permanent information carefully
SillyTavern's documentation makes the trade-off explicit: character-definition tokens that are always included leave fewer tokens for conversation history. Put only durable, behaviorally important facts in the permanent card. Move optional setting detail into activated lore and evolving events into summaries or memory entries.
Write facts in clear standalone language. Avoid three near-duplicate descriptions of the same trait. A long poetic backstory may inspire tone, but it also competes with current dialogue. Test a shorter version before assuming more text produces a deeper character.
Maintain a long conversation
- Correct errors early. Edit an incorrect stored fact where the platform permits, rather than repeating the correction in every message.
- Create periodic state summaries. Keep relationships, unresolved goals, important events, and current location; omit decorative dialogue.
- Separate stable from episodic facts. Identity belongs in the card; a recent promise belongs in memory or summary.
- Use distinctive retrieval keys. Lore triggered by generic words can flood the prompt with irrelevant entries.
- Reset deliberately. When a long chat becomes incoherent, start a new session with a clean state summary rather than dragging every failure forward.
Diagnose memory failures
- Recent events vanish: the context may be full, response allowance too large, or permanent prompt too long.
- Old facts appear at random: retrieval keys may be broad or memory ranking weak.
- A wrong fact persists: it may exist in several layers; correct the card, summary, and saved memory rather than only the latest message.
- Voice changes after a reset: preserve a concise style instruction and a few strong dialogue examples.
- Local chat becomes slow at long context: the growing KV cache and prompt evaluation may exceed available resources.
Compare platform memory controls with the companion decision guide, or improve the permanent definition with the character-card guide.