Plain-English explanation
Training and inference are the two separate phases of every AI model's life. Training happens once, or periodically when a new version is released: the model is shown enormous amounts of data and its internal parameters, its model weights, are gradually adjusted until it captures useful patterns. Inference happens every single time afterward that someone uses the model (one prompt in, one output out) without changing anything about the model itself.
When you type a message to an AI companion or click generate on an image tool, you're triggering inference. The model's weights stay frozen; the app is just running a calculation through them.
How it works
An input (a prompt, an image) is converted into a numeric form the model can process (tokens, embeddings), pushed through the network's fixed weights, and turned into an output: next-token probabilities for an LLM, a denoised latent for a diffusion model. No learning happens during this process: the checkpoint is unchanged by the request, only a new output has been computed. Inference speed and cost depend heavily on model size, the hardware running it (GPU or CPU, available VRAM), and generation settings such as inference steps or response length.
Adult-AI use
Inference is the specific cost center behind almost every practical limit a user runs into directly. Credits on a hosted app are effectively a way of metering how much inference compute a request consumes; a wait before an image or a reply appears is time spent on inference; and the choice between local generation and hosted generation is a choice about whose hardware does the inference and who pays for it. A high-end GPU running inference locally can be private and, after the upfront hardware cost, essentially free per use; a hosted service runs inference on shared server hardware and charges per use instead.
How it differs from training and fine-tuning
Fine-tuning and full training permanently change a model's weights based on new data, and typically require far more time, data, and specialized hardware than any single inference run. Inference never modifies the model, running the same prompt against the same checkpoint twice, with the same seed and settings, runs through the identical fixed weights both times, even though minor hardware or precision differences can occasionally cause tiny variation in the result.
Common uses
- Every chat reply generated by a companion app.
- Every image or video generation triggered by clicking “generate.”
- “Inference time” and per-request pricing shown on API and platform pricing pages.
Limitations and misconceptions
- Inference steps is a related but distinct, more specific term; it refers to the number of denoising iterations in a single diffusion generation, not the general inference phase as a whole.
- Faster inference doesn't necessarily mean lower quality, or vice versa: speed mostly reflects hardware, model size, and optimizations like quantization rather than a fixed, universal speed-versus-quality tradeoff.