Temporal consistency: flicker, drift and morphing
Generated clips wander because nothing in the model keeps a continuity book.
On a set, continuity survives because somebody writes it down. The script supervisor logs which hand holds the cigarette, how full the glass is, which way the collar folds, and the stills go into a continuity book so tomorrow's setup matches today's. A video model has no continuity book. It does not carry a list of objects with properties forward through a shot. What it has is a set of latents that must all look plausible together, plus attention running between them. That is enough for a lot and not enough for everything. Broad motion holds up, because a body crossing a frame is a strong and common pattern. What fails is anything the model must re-derive at each moment. The pattern on a shirt crawls because no specific pattern was ever committed to, only plausible fabric resolved a little differently each frame. A face drifts because this person is not stored anywhere. An object passes behind another and returns subtly changed, because object permanence is not a rule the model holds. VBench, which breaks video quality into 16 measured dimensions, lists temporal flickering, subject consistency, background consistency and motion smoothness as separate axes, which tells you the field treats these as distinct standing failure modes rather than one bug. Architecture trades against the problem in different ways. Cascades generate distant keyframes and then fill between them, as Imagen Video does with interleaved spatial and temporal super-resolution stages. Lumiere's authors argue that keyframes followed by temporal super-resolution makes global temporal consistency inherently hard, and generate the full duration in a single pass instead. Both approaches improve the symptoms. Neither turns continuity into something the model tracks.
Design shots around what the model cannot track. Shorter takes, less occlusion, fewer patterned surfaces, no small repeated text, and faces held long enough to read but not long enough to wander. When a face must recur across shots, consistency is bought with reference images, fine-tunes and human selection, so put that selection time in the schedule as labour. In review, watch each clip twice at speed and once at quarter speed, because flicker and texture crawl are far easier to catch slowed down, and much cheaper to catch before approval than after.
What people get wrong
Each of these is a belief worth dropping, followed by what is actually happening.
- flicker is a bug awaiting a patch. It follows from the model re-deriving detail it never committed to. Scale and architecture have reduced it considerably and have not removed it, and benchmarks still measure it as a standing dimension.
- the model tracks characters and objects from frame to frame. It keeps frames mutually plausible. Identity is not stored, which is why a face can drift inside one shot and will not match across shots by default.
- higher resolution reduces morphing. Resolution and temporal stability are separate axes, and a crisper frame often makes crawl and drift easier to see.
- interpolating between keyframes gives consistent motion. It gives smooth motion. Lumiere's authors argue that generating distant keyframes and filling between them makes global consistency harder, which is why they generate the whole duration at once.
- continuity within a shot and continuity across shots are the same problem. Within a shot, attention between latents does real work. Across shots nothing is shared, so matching has to be bought with references, fine-tunes and selection.
Terms this introduces
- temporal flicker
- Frame-to-frame instability in texture, brightness or fine detail that was never committed to in the latent.
- identity drift
- A face or character gradually changing across a shot because the model stores no persistent identity.
- texture crawl
- Patterns and fine surfaces reshuffling each frame instead of holding still on the object.
- keyframe cascade
- Generating distant frames first and then filling the gaps with temporal super-resolution models.
- temporal super-resolution
- A separately trained model that increases frame rate by generating intermediate frames.
Where this shows up in the directory
Models whose own documentation names this. It is evidence the vendor considers it relevant, not a judgement that the model is good or bad at it.
Whose job this changes
Craft roles whose page discusses this directly. Follow one to see what the person actually decides.