Upscaling and interpolation: finishing at size
Generating small and finishing large, and what invented pixels and invented frames cost you.
An optical blow-up from 16mm to 35mm gives you a bigger frame and no more information: the grain gets bigger along with everything else. Neural upscaling is a blow-up performed by a matte painter who has seen a great deal of 35mm and will happily paint in the detail the negative never held. That is the trade in one sentence, and the rest is knowing when the painter's invention is welcome.
Generating small and finishing large is standard practice for a reason. The Sora report notes that training on native sizes lets them prototype at lower sizes before generating at full resolution with the same model. SDXL ships a separate refinement model applied as a post-hoc image-to-image pass to improve fidelity. Dedicated super-resolution goes further: Real-ESRGAN is trained on synthetic degradations built by a high-order degradation model, and explicitly handles ringing and overshoot artifacts. Its priors are the degradations it was trained on, so it excels at plausible reconstruction and fails as invention when the source detail is genuinely absent. Small text, fine fabric weave and distant faces are where the painter starts making things up.
Frame interpolation is the same bargain along time. RIFE estimates intermediate optical flow end-to-end and supports arbitrary timesteps, fast enough for real-time work. FILM handles large motion with a single network trained on frames alone. Both synthesise frames that were never exposed, which has two consequences on set-shot and generated material alike. Motion blur does not rescale: intermediates between two frames blurred for a long shutter carry that smear at a shorter interval, so a 24 to 60 conversion reads with the cadence of one and the blur of the other. And flow estimation breaks at occlusion boundaries and across cuts, which is where interpolation artifacts live.
Order matters in a finishing chain: select, then any regeneration, then temporal work, then spatial upscale, then grade.
This is where you buy back resolution and cadence instead of paying for them at generation time, which is the single largest lever on generation cost. Prototype at low resolution, approve the take, then finish once. But set the deliverable honestly: an upscaled generation is not a 4K acquisition format, and generated footage arrives with a look and a grain already baked in, so it has less grading latitude than a log file. Never interpolate to hit a frame-rate spec on hero shots without checking motion on a large screen.
What people get wrong
Each of these is a belief worth dropping, followed by what is actually happening.
- upscaling recovers detail. It reconstructs plausible detail using priors learned from training degradations, which is restoration where the information is present and invention where it is not.
- a 4K upscale is a 4K deliverable. It meets a pixel count; it does not meet the detail, edge behaviour or grading latitude of footage acquired at that size.
- interpolation makes motion look better. It makes motion smoother, which is a different thing. Frames synthesised between long-exposure frames keep the original blur, so cadence and blur stop agreeing.
- interpolation and upscaling are interchangeable finishing steps in any order. Temporal work before spatial work generally holds up better, because upscalers amplify the flow artifacts interpolation leaves at occlusion edges.
- more finishing passes always improve the frame. Stacked super-resolution and sharpening compound ringing and overshoot, the exact artifacts a blind super-resolution model such as Real-ESRGAN is trained to suppress rather than to create.
Terms this introduces
- Super-resolution
- Increasing pixel dimensions by predicting the detail a higher-resolution version of the image would plausibly contain.
- Blind super-resolution
- Upscaling when the degradation that damaged the source is unknown, trained on synthetic degradation pipelines instead.
- Detail hallucination
- Invented high-frequency content that looks correct in isolation but does not correspond to anything in the source.
- Frame interpolation
- Synthesising frames between existing ones, usually by estimating intermediate optical flow, to raise frame rate or slow motion.
- Cadence
- The rhythm of motion set by frame rate and shutter, which stops matching the picture when frames are synthesised after the fact.
Where this shows up in the directory
Models whose own documentation names this. It is evidence the vendor considers it relevant, not a judgement that the model is good or bad at it.
Whose job this changes
Craft roles whose page discusses this directly. Follow one to see what the person actually decides.