Sampling and guidance: the two dials that shape a generation
How many stops the model makes on the way, and how hard it steers toward the prompt.
Sampling is a drive from noise to picture, and you choose how often to stop and check the map. Each stop is one step: the model looks at the current noisy latent, predicts the noise in it, and the scheduler decides how far to move and where the next stop sits. The scheduler is the route plan. DDPM's original formulation stopped many hundreds of times. DDIM built a non-Markovian process with the same training objective but a much faster reverse path, reporting samples 10 to 50 times faster in wall-clock terms and framing the choice as compute traded against quality. Diffusers ships a shelf of interchangeable schedulers that run on the same trained weights, which is why one prompt and one seed give different pictures under different samplers. Past a certain number of stops the route stops changing. Remaining steps move the latent a negligible distance, so they buy render time and not quality, and where that plateau sits depends on the scheduler and the model. Worth finding empirically rather than defaulting high. Guidance is how hard you steer toward the prompt. Classifier-free guidance runs the model both with and without the conditioning and pushes the result away from the unconditional prediction. The paper presents this as a trade between sample fidelity and diversity, in the same spirit as low temperature sampling. Over-steer and you overshoot: crushed blacks and blown highlights, oversaturated colour, hard rimmed edges, composition collapsing into one obvious reading of the prompt, and every seed returning nearly the same shot. Distillation shortens the drive. Progressive distillation halves the step count repeatedly, from thousands down to four. Consistency models map noise to data in a single step, and latent consistency models bring that to Stable Diffusion at two to four steps.
Steps and guidance are worth calibrating once per model, then locking. Find the step count where more steps stop changing the frame, and the guidance value where the prompt reads without contrast blowing out, and write both into the shot's parameter record next to the seed and model version so a revision months later starts from the same place. Use a distilled few-step model for the exploration pass, where the point is to see twenty options, and the full model for the approved shot. That split usually buys more looks per pound than tuning prompts does.
What people get wrong
Each of these is a belief worth dropping, followed by what is actually happening.
- more steps always improves the image. Sampling converges, and past the plateau the frame stops changing while the render time keeps climbing.
- the scheduler is a quality setting where one option is simply better. It is a different route through the same trained weights. Different schedulers give genuinely different pictures from the same seed, and none is a strict upgrade.
- higher guidance means better prompt adherence. Only up to a point. Beyond it, contrast and saturation blow out, composition collapses to one reading, and variety across seeds disappears, which is exactly the fidelity-versus-diversity trade the guidance paper describes.
- few-step distilled models are just fast versions of the same model. They are separately distilled models with their own look, and they often fix guidance internally, so a control you relied on may not be exposed.
- the seed is the only thing that decides the output. Scheduler, step count, guidance value and model version all change the result, so all of them belong in the record if a shot has to be reproduced.
Terms this introduces
- step
- One pass of the model plus one scheduler update, moving the latent partway from noise toward a picture.
- scheduler
- The rule that decides step sizes and how each noise prediction is applied; also called the sampler.
- classifier-free guidance
- Running the model with and without conditioning and pushing away from the unconditional prediction to strengthen prompt adherence.
- guidance scale
- How strongly that push is applied, trading prompt fidelity against variety.
- distillation
- Training a student model to reproduce a many-step sampler's output in far fewer steps.
Where this shows up in the directory
Models whose own documentation names this. It is evidence the vendor considers it relevant, not a judgement that the model is good or bad at it.