From prompt to embeddings
A tokenizer first divides prompt text into IDs from a fixed vocabulary. The text encoder processes that sequence and produces contextual vectors. The generator consumes those vectors through cross-attention, joint attention, adaptive normalization, or another documented conditioning path.
The encoder can produce a vector for each token plus one or more pooled summaries. A model family may use both forms for different conditioning signals. Padding, truncation, special tokens, and maximum sequence length determine which prompt text reaches the encoder.
Why models use different encoders
Stable Diffusion 1.x uses a CLIP text encoder. SDXL uses two encoders. Stable Diffusion 3.5 Large documents two CLIP encoders plus T5-XXL, and FLUX.1 uses CLIP and T5 components. Each encoder contributes representations learned from its own training objective and vocabulary.
More encoders can increase conditioning capacity while also increasing downloads, RAM, VRAM, and prompt-format complexity. It does not mean every word receives equal influence or that the generator can render every relationship the encoders represent.
Compatibility and local inference
A checkpoint may bundle or reference specific text-encoder weights. Quantizing, lowering precision, or offloading an encoder can reduce GPU memory, but the loader must support that architecture and data type. Large T5 components can dominate startup memory even though the denoiser performs the repeated image steps.
Swapping encoders is not generally equivalent to changing a VAE and can invalidate learned prompt relationships. The tokenizer and encoder weights must match, and a pipeline configuration must know which hidden states, pooled outputs, and attention masks to use.
Adult-AI relevance and limits
Text encoders influence prompt understanding, tags, character names, style associations, and how much text fits before truncation. Illustration models may learn tag conventions on top of an inherited encoder, so the same visible prompt can behave differently across fine-tuned descendants.
Text encoders do not enforce consent, establish factual knowledge, or guarantee that a prompt detail appears in the output. Hosted systems may change encoders without exposing the component, which limits reproducibility across service updates.