How Image Models Work
From pure static to a finished picture in seconds — meet diffusion, the surprisingly simple idea behind modern image models.
Here is a fun secret: every AI image you have ever admired started life as pure static — the visual equivalent of an untuned TV. A diffusion model's entire job is to stare at that noise and gradually sculpt a picture out of it, guided by your words. Michelangelo claimed the statue was already inside the marble; diffusion models take that idea literally.
Training: learning to un-ruin images
During training, the system takes billions of real photos and deliberately ruins them — adding a little noise, then more, then more, until each image is unrecognizable static. The neural network's homework is to reverse each step: look at a noisy image and predict exactly which noise was added, so it can be removed. Repeat this across billions of examples from the training data, and the network becomes a world-class *de-noiser* — a skill that turns out to be the key to creating images from nothing.
Generation: running the film backwards
- Start from a canvas of pure random noise (which noise exactly is set by the seed).
- A text encoder converts your prompt into numbers the model understands.
- The model predicts which noise to remove so the image drifts toward your description.
- Repeat for roughly 20-50 denoising steps — each pass makes the image sharper and more coherent.
- Decode the result into the final pixels you see on screen.
Where the words come in: image-text pairs
The words work because the model trained on billions of image-caption pairs. Seeing "golden hour" written under millions of warm, low-sun photos taught it what those words *look like*. This is why prompt vocabulary matters so much: terms photographers and artists actually use are richly represented in the data. That is the entire premise of text-to-image generation — and of the Style and Aesthetic Language lesson later in the Academy.
Latent space: the compressed workshop
One more trick makes it all fast: most models do the denoising not on millions of raw pixels but in latent space — a compressed representation of the image, dozens of times smaller. Only at the very end does a decoder expand it back into a full-resolution picture. It is the difference between sketching on a napkin and repainting a billboard for every draft — and it is why your image takes seconds, not hours.
| Knob | What it controls |
|---|---|
| Seed | Which random noise you start from — same seed + same prompt gives a near-identical image |
| Sampling steps | How many denoising passes run — typically 20-50 |
| Guidance (CFG) | How strictly the model follows your prompt versus improvising |
| Aspect ratio | The shape of the canvas: 1:1 for social posts, 16:9 for cinematic frames |
Feel the denoising steering
Macro photograph of a dew-covered spider web at sunrise, backlit, glistening droplets, soft bokeh background, extremely detailed, 85mm lens
Every phrase here steers the denoising in a measurable direction: "macro" sets scale, "backlit" sets light, "85mm lens" sets optics. Remove one and watch the result shift.
Watch noise become art in about ten seconds. Generate an image