Skip to main content
Module 1: How it actually works

The model writes a program, not a video

Opus 5.5 outputs text, so every "AI video" is really a deterministic program it wrote that a headless browser renders frame by frame.

Here is the fact that makes everything else make sense: Claude Opus 5.5 takes text and images in and puts text out. It cannot emit an MP4. So every one of those "AI-made" motion videos is really a program the model wrote, which something else turns into frames.

The engine of the whole trend is determinism. Rather than animate against a live clock, the model writes one function that paints the exact frame for any moment in time. We will call it seek(t): give it a time in seconds, it draws that frame. A headless browser calls seek(t) hundreds of times — 900 times for a 15-second clip at 60fps — screenshots each frame, and ffmpeg stitches them into a video.

Why determinism matters so much:

  • Reproducible. No timers, no Math.random, no state carried between frames. Frame 812 is the same every single run. That means you can render just the seconds you changed, and a tweak is a one-line edit plus a re-render — not a re-roll of a black-box video model.
  • Seekable. Because any frame is a pure function of t, you can jump straight to the fast moment at 4.2s to inspect it, or render a contact sheet of one frame per beat without simulating everything before it.
  • Ownable. The output is code you can read, diff, and reuse. A video model gives you pixels you cannot edit; this gives you a program you can.

This is not a fringe trick. It now has a name — "video as code" — and real tooling. HeyGen open-sourced HyperFrames (Apache-2.0) to do exactly this from HTML; React developers use Remotion to do it from components. Both capture a deterministic program frame-by-frame in headless Chrome and encode with ffmpeg. Different front doors, same idea underneath.

One honest note on why Opus 5.5 is good at this: Anthropic's own launch described a test where several models built a game from one prompt, and Opus 5.5 scored highest on the strength of its graphics and polish. Writing a clean, animated visual program is squarely in its wheelhouse. But the model is still only writing code — which means your job, for the rest of this course, is to build the machine that runs and improves that code.

Try it

In your own words, write the one-sentence pipeline that turns a prompt into an MP4 (model -> program -> browser -> frames -> ffmpeg). Then list two things determinism buys you that a black-box video model cannot.

Stay in the loop

Enjoying the free lessons? Get an email when we publish new courses and updates — no spam, unsubscribe anytime.

Discussion (0)

Ask a question or share what worked for you. Comments are reviewed before they appear.

Log in to join the discussion and ask questions about this lesson.

No comments yet. Be the first to start the discussion!