Character and set consistency: the weakest link

Every generation is a fresh casting session. You are not shooting the same actor on Tuesday that you shot on Monday; you are recasting the part each time from a description, and hoping the audience does not notice the new jawline. Reference images are the casting brief. A subject fine-tune is a stand-in trained until they pass for the actor. Human selection is the casting director throwing out the ones that do not match. Nothing in the pipeline is the actor.

The real mechanisms sit on a ladder of cost. At the cheap end, identity adapters: IP-Adapter adds a decoupled cross-attention path so an image prompt is read alongside the text prompt, with roughly 22 million adapter parameters and the base model frozen, which is why it composes with structure controls. Above that, subject fine-tuning. Textual Inversion learns a new pseudo-word in the frozen model's embedding space from three to five images, so the concept becomes something you can write into a sentence. DreamBooth goes further, fine-tuning the model to bind a unique identifier to a subject from a handful of images, with a prior-preservation loss added specifically to stop the model collapsing the rest of the class onto your subject.

Cross-shot continuity is the weakest link in the whole pipeline, and the field's own evaluation admits it: VBench includes subject identity inconsistency among its sixteen dimensions of video quality. The Sora report claims the model can hold a character's appearance across multiple shots within a single sample. That qualifier matters. Within one generation the model has a shared context; across two generations it has none. Same face, same wardrobe, same set dressing, same light direction, same eyeline: each is re-derived from scratch, and each is a place a shot fails.

What people get wrong

Each of these is a belief worth dropping, followed by what is actually happening.

• the model remembers the character from the last generation. It has no persistent memory of your cast or set between generations; continuity comes from what you re-supply as conditioning each time.
• one good reference image is enough. Single references pin a look from one angle and one light; identity fails first on profile, extreme angles, and lighting far from the reference, which is why fine-tuning methods use small sets rather than single frames.
• a DreamBooth-style fine-tune is strictly better than an adapter. A fine-tune changes the weights and can degrade everything else the model knew, which is why DreamBooth carries a prior-preservation loss; an adapter such as IP-Adapter leaves the base model frozen and stays compatible with other controls.
• consistency means the face. Wardrobe detail, set dressing, prop position, hair, light direction and eyeline all drift independently, and audiences read a change of light direction across a cut as fast as they read a change of face.
• this is a solved problem in current models. Cross-shot continuity remains the weakest part of the pipeline and is currently bought with references, fine-tunes and human selection rather than with model memory.

Terms this introduces

Identity adapter

A small trained module that lets a frozen base model read a reference image of a subject as a prompt, IP-Adapter being the common example.

Subject fine-tuning

Updating model weights, or an adapter on them, from a small image set so a specific person, object or set can be called up by name.

Textual Inversion

Learning a new embedding, effectively a new word, for a concept while leaving the model itself unchanged.

Prior preservation

A training term that keeps a fine-tune from overwriting the model's general knowledge of the class your subject belongs to.

Character sheet

A prepared set of reference frames covering angles, expressions, wardrobe and lighting, used as conditioning across a sequence.