Plain-English explanation
Training a large model from scratch (teaching it language or image generation from nothing) takes enormous amounts of data and compute, and is done once by whoever originally builds the base model. Fine-tuning starts from that already-capable base model and nudges it further, using much smaller, more targeted training data, so it gets better at (or specialized for) something the base model wasn't already great at: a particular character, an art style, a writing voice, a narrower domain, or a specific behavior. It's the difference between building a car from raw materials and modifying an existing car for a specific purpose.
Common approaches
"Fine-tuning" is a category, not one specific method: the actual mechanics vary a lot depending on how much of the model gets updated and how the result is stored:
- Full fine-tuning. Every weight in the model (or a large portion of them) is updated through further training, typically producing a new full-size checkpoint. This gives the most flexibility but needs the most compute, memory, and data, and carries the highest risk of the model "forgetting" prior capabilities.
- LoRA. A parameter-efficient method that freezes the base model and trains a small set of add-on weights instead, producing a lightweight adapter file.
- DreamBooth. A fine-tuning method specifically aimed at teaching a diffusion model a particular subject from a handful of images, usually by updating the model's own weights (sometimes saved as a LoRA instead in lighter variants).
- Textual Inversion. A minimal-footprint approach that only trains a new text embedding, leaving the model's own weights untouched: arguably the lightest form of customization, sitting right at the edge of what counts as "fine-tuning" the model itself versus just teaching it a new word.
Adult-AI use
Fine-tuning is the umbrella activity behind almost every customized AI companion character and image-generation "character" people build locally. Someone downloading a base checkpoint and applying a character LoRA is doing fine-tuning-adjacent work, just via the lightest available method; someone training a dedicated DreamBooth checkpoint is doing heavier, full fine-tuning work. It's also how many uncensored or NSFW-capable local models come to exist in the first place: a base model fine-tuned on a dataset that shifts its behavior away from the more restrictive defaults of its original release.
Limitations and misconceptions
- "Fine-tuning" alone doesn't specify the method. When someone says a model was "fine-tuned," that could mean anything from a full retrain to a small LoRA: the specific technique matters for compute cost, file size, and how permanently it changes the model.
- More training isn't automatically better. Overfitting (memorizing training data too literally) and catastrophic forgetting (losing prior general capability) are both real risks, and get worse with too many training steps or too little/too narrow data.
- Fine-tuning is not the same as prompting. Fine-tuning changes something about the model's stored weights or embeddings; a prompt (including a system prompt) only steers a fixed, unchanged model at generation time.
Privacy and safety considerations
Fine-tuning a model on a real person's photos, writing style, or voice without their consent raises the same consent and likeness concerns as any other form of AI customization built around a real identifiable person: see the LoRA and DreamBooth entries for how those concerns apply to specific techniques.