Control is a stack, not a slider
An image-control workflow typically has at least four layers: a source or preprocessor produces a condition; a control network or adapter encodes it; the base denoiser integrates the signal; and interface settings determine strength and timing. A pose map, depth map, mask, and reference image carry different information and should not be treated as interchangeable “image prompts.”
Conditioning is the umbrella concept. ControlNet, IP-Adapter, inpainting, image-to-image, and model-specific reference systems implement different paths.
ControlNet and image adapters
ControlNet’s paper describes a trainable copy of pretrained diffusion-model blocks connected through zero-initialized convolutions. It can learn spatial controls such as edges, depth, segmentation, and pose while leaving the production model locked. IP-Adapter instead uses image features as an additional cross-attention condition alongside text, aiming for image-prompt compatibility without retraining the base.
Modern families may ship native Canny, depth, or editing variants that are not literally ControlNet. The user-facing result may look similar while the learned component and input path differ.
Composition and compatibility
Multiple controls can be combined, but strengths interact and every component adds memory and another failure surface. A preprocessor can misread anatomy; a pose can conflict with the prompt; a reference adapter can over-copy composition; a mask edge can reveal a seam. Start with one documented control, verify it, then add another.
Control models and adapters target specific base families and sometimes exact releases. Tensor shape compatibility does not ensure useful behavior. ComfyUI exposes composition as a graph; Forge, AUTOMATIC1111, and InvokeAI package controls through panels or canvases.
Character consistency, safety, and limits
Reference conditioning can help preserve fictional-character consistency across scenes, while pose/depth controls stabilize structure and inpainting repairs a local region. None guarantees identity. For sensitive images, local processing reduces platform upload exposure only when preprocessors, nodes, and storage remain local.
Do not use control stacks to sexualize an identifiable person without consent. A generated edit is synthetic media, not evidence of the subject’s body or behavior. Strong reference adherence can increase likeness risk, so source permission and access control matter.