← All posts

AI video in a content pipeline: edit the frame, don’t invent the clip

Runway Aleph in production: shot manifests, one op per pass, preserve clauses, human QC, and prompt caching.

AI video in a content pipeline: edit the frame, don’t invent the clip
Contents

In brief

A two-person team could not hand-cut short product videos on schedule. A Dev.to write-up covers wiring Runway Aleph: the model makes a bounded edit to real footage instead of inventing a new clip. Text-to-video demos well and fails continuity in production.

What happened

Generate-first tools produce independent clips: “the same” product is a different product each time. An in-context model takes source video and a scoped change — remove a price tag, keep box, surface, lighting.

Pipeline: raw footage → JSON shot manifest → edit passes → human QC → publish. The manifest was underestimated: ops are replayable, one per entry. Batching “remove / relight / add a plant” ruins all three and cannot be debugged.

Explicit preserve clauses stop quiet background re-renders. Similarity auto-QC catches gross failures and misses semantics: fingers, gibberish signage, broken reflections. A content-addressed cache on source + prompt cut more than half of iteration spend.

Why it matters

The win is not “impossible became possible.” Expensive work got cheap enough to attempt. Real cost sits in full-resolution review, not generation seconds.

In practice

  1. Add one stage first (object removal), don’t rebuild the whole pipeline.
  2. One operation per pass; batch only if you accept not knowing which clause broke the frame.
  3. Always state what to preserve, not only what to change.
  4. Human QC on risk regions at full resolution, not in a preview pane.
  5. Cache identical source+prompt pairs; benchmark including review time.

Takeaway

AI video in content works as controlled editing, not a factory of random clips. Manifests, narrow passes, and human QC matter more than the model brand.