Diffusion Model

A diffusion model makes data by learning to undo noise. Training adds Gaussian noise to real samples, step by step, until only noise is left. The model learns to predict and remove the noise at each step.

Generation runs that process backward. Start with random noise. Apply the learned denoiser again and again. Each step is a small correction. Hundreds of small corrections turn noise into an image, a clip of audio, or a video frame. That slow refinement is why the samples look sharp.

DALL-E 2, Midjourney, Stable Diffusion, and Imagen are diffusion models. They generate photos from text. Training is more stable than GANs, and sample quality is usually higher. The network is often a U-Net. It sees the noisy image, the timestep, and optional guidance such as a text embedding. Classifier-free guidance trades variety for closer match to the prompt.

The main cost is speed. One image can take hundreds or thousands of network passes. DDIM, progressive distillation, and consistency models cut the step count. Diffusion now dominates image, video, and audio generation. It is a denoiser run in reverse, not a single-shot painter.

The forward process is a fixed noise schedule. The reverse process is the learned part. Guidance, step count, and sampler choice change the look as much as the prompt. A better prompt does not fix a bad sampler. DDPM (2020) is the usual citation for the noise-then-denoise image models behind Stable Diffusion and DALL-E 2.