Plain-English explanation
An LLM generates text by predicting what's statistically likely to come next, not by looking facts up in a database and confirming they're true. Most of the time this produces accurate, useful output, because plausible-sounding text and true text overlap heavily in a well-trained model. But sometimes the model produces a statement that's fluent, specific, and stated with total confidence, and simply wrong, or invented outright. That's a hallucination: the model isn't lying in the human sense, it doesn't have a concept of truth to violate, it's generating what its training makes statistically likely, and in that particular case that output happens to be false.
How it works
OpenAI's own research on the topic frames hallucinations as an expected consequence of how these models are trained and evaluated: next-token prediction rewards producing plausible, fluent text, and benchmarks that penalize uncertain or "I don't know" answers can push a model toward confidently guessing rather than admitting it doesn't know something. Hallucination isn't a rare glitch limited to obscure edge cases: it's a structural property of generating text via next-token prediction, and it shows up more when a model is asked about something outside or thinly covered by its training data, when a query is ambiguous, or when a conversation has grown long enough that earlier details have become unclear or contradictory to the model.
Adult-AI use
Inside a companion or roleplay conversation, hallucination usually shows up as a character confidently "remembering" something that never happened, contradicting an earlier established detail, or inventing a fact about itself or the relationship on the spot. Because companion interactions are conversational and emotionally framed rather than fact-checkable in the way a search query is, these hallucinations can be harder to notice than a factual error would be in, say, a research assistant: a confidently wrong "remember when you told me..." can feel like real continuity even when it's fabricated.
How it differs from a memory failure
These are easy to conflate but distinct: a memory failure is the model or app failing to retain or retrieve a real fact (see character memory and long-term memory): the honest version is the character simply not knowing or not recalling something. A hallucination is the model inventing a specific, confident, wrong answer instead of acknowledging uncertainty. Both can happen in the same conversation, and a memory system that retrieves the wrong stored detail can itself trigger what looks, to the user, like a hallucination.
Limitations and misconceptions
- It's not unique to "bad" or small models. Every current LLM hallucinates under some conditions; larger and more capable models generally hallucinate less often on well-covered topics but are not immune.
- Confidence isn't evidence. A hallucinated statement is typically delivered with exactly the same fluent, assured tone as an accurate one: the model's phrasing gives no reliable signal about whether a specific claim is true.
- RAG and grounding reduce but don't eliminate it. Giving a model relevant retrieved information lowers hallucination risk but doesn't guarantee the model will use it correctly.