Plain-English explanation
Think of the context window as the model's short-term working memory for a single request: everything it can "see" while generating a reply, measured in tokens rather than messages or characters. It includes the hidden system prompt, as much of the prior conversation as fits, and the response being generated, all counted against the same limit.
A useful way to picture it: if the context window is like a whiteboard of fixed size, everything relevant to the current reply has to physically fit on that whiteboard. Once it's full, something has to come off before anything new gets added.
How it works
Context window size is a property of the specific model, not something a user or app can arbitrarily increase: it's set by the model's architecture and how it was trained. Sizes vary widely between models and have grown substantially over time: many mainstream models now offer context windows in the hundreds of thousands of tokens, with some frontier models supporting a million or more, though very large context windows don't necessarily mean the model uses every part of that window equally well.
In a chatbot, the model has no memory between separate requests: each time you send a message, the app re-sends the system prompt plus as much prior conversation as it can fit, up to the context window limit, and the model generates a response based only on the text in that window. Anything that falls outside it is invisible to the model unless it's been captured elsewhere, a limitation that character memory, long-term memory, and RAG systems exist to work around.
Adult-AI use
Context window is the reason a long roleplay or companion conversation can start to "forget" earlier details even without any dedicated memory failure: once the conversation's token count exceeds the model's context window, the app has to trim, summarize, or otherwise drop older messages to keep fitting within the limit. Users on companion apps often notice this directly as characters losing track of established facts, names, or plot details from earlier in a long chat. It's also a cost and performance factor for platforms: processing a larger context window is more computationally expensive, which shapes how much conversation history a given app chooses to keep active per message.
How it differs from character memory, long-term memory, and RAG
These are commonly confused because they all relate to an AI "remembering" things, but they're distinct mechanisms:
- Context window is the hard technical ceiling on how much text a model can process per request: it's a limit, not a feature, and it resets its relevance once a conversation moves past it.
- Character memory is the product-level feature built on top of (or around) the context window to make a character seem to remember you.
- Long-term memory specifically means storing information outside the context window so it survives across sessions, rather than just being whatever currently fits in the window.
- RAG is one common technique used to implement long-term memory, retrieving relevant stored information and inserting it back into the context window right before generating a response.
Limitations and misconceptions
- A larger context window isn't the same as better memory. Even models with very large windows can weight information unevenly across a long context, and a big window doesn't by itself give a model memory across separate conversations.
- Tokens, not words or messages, are what count. A short passage in a token-dense language or with unusual formatting can use up more of the window than its word count suggests.