Yuval Avidani
Author
The folks at HeyGen just dropped a tool called HyperFrames that says something almost provocative: if we know how to write a webpage, we already know how to direct a video. HyperFrames is an open-source engine that takes a plain HTML page and turns it into a precise MP4 video file, frame by frame — for anyone who wants the machine, or an AI agent, to produce the video by itself.
Let's break this down slowly, because behind this simple idea hides a problem we all know. Let's start from the basics, because I can't assume anything: HTML is the language every website we open is built with — it tells the browser "here's a title, here's an image, here's a button." Think of it like a blueprint for a room: not the furniture itself, but the instructions for where everything sits. Alongside it there's CSS, the styling layer (colors, sizes, motion), and JavaScript, the code that moves things around. Almost every developer in the world knows this trio of tools, and that's exactly the point.
The situation: we want the machine to produce video — but the tools are built for fingers
Turns out that when we ask an AI agent today to make us a video — a product teaser, a social clip, an animated explainer — we get stuck. Regular video editors like Premiere or CapCut are built for human fingers dragging clips on a timeline; there's no "text you can hand to a model to write." And on the other side, the frameworks that do generate video from code demand knowledge the model doesn't really have a grip on.
This is where HyperFrames' bet comes in. The exciting part is that an agent who knows how to write HTML becomes, almost for free, a capable video editor — because the video's format is literally the language it already speaks fluently. You don't need to teach it a new tool; you just need to let it write what it's already best at.
The obstacle: getting video out of a browser is insanely hard
Here we need to stop and be fair, because the problem is real and hard. A browser simply wasn't built to produce video. It runs "in real time" — if the computer gets busy for a moment, one frame will take a bit longer, and two different machines will spit out different results. For video we need something called determinism: the same input has to produce exactly the same MP4, every time, on every machine (that's what "deterministic" means — unambiguous, no surprises and no randomness).
The tool that solved this first is called Remotion, and it deserves credit. Remotion is a framework where you build video using React — a popular code library for building interfaces — and thousands of companies have been producing video with it in production for years. That's a different, excellent approach. But the HeyGen team discovered something interesting when they tried running AI agents on it: "the more guardrails we added, the safer and more repetitive the output became. When we went back to plain HTML, the creativity came back." The reason lies in the data: React is a small slice of what models are trained on, while HTML is everywhere on the internet — so the model is simply freer when writing HTML.
The answer: "seek, not play" — and how it actually works
So how do you get a stable video out of a wild browser? The most important trick is called "seek, don't play." Instead of letting the video "run," the engine moves time by hand: jump to moment 0, take a screenshot, jump to the next frame, shoot again — and so on to the end. Every frame is captured separately and under full control, and then a tool called FFmpeg (the Swiss army knife of video processing) stitches all the images into one smooth file.
Every video is essentially one index.html file — they call it a "composition" — and inside it, every element gets two small attributes: data-start (when to appear) and data-duration (how long to stay on screen). That's it. A title that enters at second 1 for 4 seconds, a background video that starts from zero, a soundtrack — they all sit on the same timeline, in a language every model understands. There's also an elegant solution to an annoying problem: browsers don't decode video the same way on every machine, so HyperFrames simply breaks down every video-within-a-video into images using FFmpeg before the shoot, and injects them back — so the result is always identical.
And here's the part I love most: to make sure the preview matches the final output exactly, HyperFrames signs the package with a digital fingerprint (SHA256), so what we see in the preview is literally the code that produced the MP4, not some approximation of it. Animations run through GSAP (the web's popular animation library) by default, and it's all completely free under an Apache 2.0 license — a permissive open-source license, no per-render fees and no limit on team size.
What it can actually do, and how to get started
So what are we building with this? Things I'm already playing with myself: product teasers, short clips for social, animated explainers, data visualizations, and captions and text overlays on top of existing video. And the coolest part? The three videos you're about to see — we built them right here in the newsroom with HyperFrames itself, using our own data. This is an article about HyperFrames, so it's only fair it demonstrates itself — an opening card, the graph from the article, and the render command that produced exactly what you're seeing. And if that hasn't convinced you yet — here's real 3D: a 3D iPhone and MacBook (Three.js) with our newsroom on screen, also rendered in HyperFrames:
And three more examples, each showing a different capability:
And the workflow with an agent feels almost magical: we ask Claude Code "make me a ten-second product intro," and it sets up a composition, wires up the tracks, writes GSAP animations, runs a preview — and renders. All without touching a video editor even once.
And getting started really is just three commands. All you need is Node.js version 22 or higher and FFmpeg installed, and then:
And to understand where this tool stands against Remotion — and when Remotion is actually the right choice — I broke down both approaches side by side, including the number that matters most when you're running thousands of renders: cost.
Bottom line: not "better," just different — and built for the age of agents
In my view, and it's important to say this plainly: HyperFrames isn't "better than Remotion." It's a different approach with one clear, sharp bet — if the future is AI agents generating video on their own, then the format needs to be the language they're most fluent in, and that's HTML. And there's a price to that, one we shouldn't hide. Rendering on a single machine is slow: in one independent reviewer's test, a 30-second 1080p video took about three minutes locally, versus roughly 25 seconds on Remotion Lambda in the cloud. The full deterministic magic only works on Linux (Mac and Windows fall back to an approximate method), the ecosystem is still thin compared to the massive React world, and the visual editor isn't mature yet. If you already have a working Remotion pipeline and an experienced React team — there's no reason to switch.
But the question I want to leave us with is a completely different one: if tomorrow every agent knows how to write HTML, and HTML is already video — how long until the words "editing a video" sound to us as outdated as "developing film"?