Plain-English explanation
VRAM is the GPU's working space. A model must place the data needed for computation somewhere close enough to the processor to be useful, and GPU memory is much faster for that purpose than repeatedly fetching everything from system RAM or storage. This is why local-AI guides commonly describe a model as requiring a certain amount of VRAM rather than only naming a GPU generation.
Capacity and compute speed are separate. A GPU can be fast but unable to fit a large model or high-resolution workflow; another can fit it but run slowly. Free memory reported before launch is also not the whole story because frameworks reserve memory and temporary tensors grow during a job.
What occupies VRAM
- Model weights: usually the largest fixed allocation, reduced by quantization or lower-precision formats.
- Activations and workspace: intermediate values whose size depends on architecture, batch size, image resolution, and implementation.
- KV cache: stored attention keys and values that grow with language-model context length and generation.
- Decoded media and auxiliary models: VAEs, text encoders, ControlNets, upscalers, and video frames can add substantial peak demand.
Why it matters in local adult AI
Local tools such as AUTOMATIC1111, ComfyUI, KoboldCpp, and LM Studio expose the practical effects of memory limits. Image size and batch count matter for diffusion; model size, precision, and context window matter for chat. Video workflows can be especially demanding because several frames or latent states may be processed together.
VRAM versus system RAM
System RAM belongs to the CPU and is usually larger but farther from the GPU. Model offloading can move weights or caches between the two to make an otherwise oversized workload run, at the cost of transfers and often substantial latency. “Shared GPU memory” reported by an operating system does not perform exactly like dedicated VRAM.
Common misconceptions
- A published requirement is not universal. Drivers, precision, attention implementation, batch size, and extensions alter peak use.
- Allocated is not identical to actively used. Framework caching allocators may retain blocks for reuse.
- Out-of-memory errors can be transient. Fragmentation or another process may consume the margin.
- More VRAM does not improve model quality by itself. It permits larger or less compressed workloads; model and settings still determine output.
Technical context
The local stack dossiers show why image tensors and auxiliary models scale differently from LLM weights and KV caches.