Plain-English explanation
Inpainting is targeted editing: you draw or select a mask over the part of an image you want to change (a hand, a background object, a piece of clothing) and the model fills that masked area in based on a prompt, while leaving the unmasked parts of the image as they were. It's the AI-image equivalent of a precise touch-up tool rather than a full repaint.
Hugging Face's Diffusers documentation frames the relationship to image-to-image directly: both tasks generate a new image that resembles the input, but inpainting only transforms the masked area while image-to-image can change the whole picture. Inpainting is the more precise tool; image-to-image has the broader scope.
How it works
- Provide a base image and a mask. The mask marks which pixels are open for regeneration (conventionally white) versus which must be preserved (black).
- Write a prompt describing the replacement content. The prompt only needs to describe what should appear in the masked area, not the whole scene.
- The model denoises within the mask. Noise is added to the masked region and progressively removed, guided by the prompt, over a number of inference steps.
- The result is blended back in. Many tools blur the mask edges or crop-and-upscale the masked region ("padding mask crop") to smooth the transition between generated and original pixels.
Purpose-built inpainting checkpoints (fine-tuned specifically on masked-image tasks) generally produce cleaner, more natural transitions than using a general text-to-image checkpoint for the same job, though a general checkpoint can better preserve the exact unmasked area.
Adult-AI use
Inpainting is the tool people reach for when a generated character image is almost right but one detail is broken: a common failure mode in AI-generated hands, faces, or specific anatomy. Rather than regenerating the whole image and risking losing everything else that worked, a user masks just the flawed region and regenerates that. It's also used for controlled edits: swapping an outfit, changing an accessory, or altering a background without touching the character's face or pose, which matters a lot for maintaining a consistent character across a set of images.
Common uses
- Fixing malformed hands, faces, or other anatomy in an otherwise good generation.
- Swapping clothing, accessories, or background elements.
- Removing an unwanted object or artifact from an image.
- Combined with ControlNet for more structurally precise edits, like matching a specific pose within the masked area.
How it differs from image-to-image and outpainting
Inpainting is scoped to a user-selected mask inside the existing image, everything outside that mask should stay the same. Image-to-image has no mask and can alter the entire image, just guided by how similar to the original you want the result. Outpainting extends an image: instead of masking a region inside the image, it masks a border of new canvas outside the image and fills that in, extending rather than editing the original.
Limitations and misconceptions
- Inpainting isn't perfectly non-destructive, depending on the checkpoint and settings, pixels near the mask edge can shift slightly, which is why blur and mask-cropping options exist.
- A poorly drawn or too-tight mask often produces visible seams; giving the model a bit of surrounding context in the mask usually helps blending.
- Inpainting fills the mask according to the prompt and the model's priors; it doesn't "know" what was originally supposed to be there unless that's inferable from the surrounding image.