Johnattan's Laboratory
← All experiments

A video studio where the model writes the program that draws every frame

I wanted to make animations and videos with AI, but the methods were scattered across repositories and posts, and video models misspell text. So I built a studio with a knowledge base, a weekly scout that keeps it current, and a tool chain for code-drawn and generative video.

  • Video
  • 3D
  • Automation
  • Tools

Updated19 Oct 2026

The problem

I wanted to learn how animations and videos are made with code and AI, and to use that knowledge to make real pieces. After Claude Opus 5.5 launched on 2026-09-22, a wave of films drawn entirely in code appeared, but the know-how was spread across hundreds of GitHub repositories and posts, most of it unverified.

What was out there

Curated lists collected the films, but almost everything in them is the creator's own claim. Only repositories with source code let you check how a film was really made, and blog roundups repeat each other and add errors. Video models are the other route, but they misspell lettering on signs and screens, and every take costs generation credits.

What I built

The Motion studio has two halves.

A knowledge base on how animations and videos are made, in three tracks: films rendered to video, motion that runs live on web pages and slides, and generative or hybrid video. It holds technique notes (storyboards and timing, painted 2D, Canvas films and frame capture, sound in code, 3D with Three.js and Blender), a catalogue of other people's work with what was verified in source and what is only claimed, a tools catalogue, prompts that worked with 21 patterns behind them, close studies of the most instructive films, and my own lessons.

A studio where I apply that knowledge. The core idea is that the model writes a program that draws every frame as a function of time, instead of generating the video itself. A headless browser (or Blender, or a Node script) renders the frames, the soundtrack is synthesised in code from the same cue list, and ffmpeg joins them into an MP4. Generated material (stills, clips, voices, music) comes in when a shot needs it. Narration starts with free or included voices (Mistral Voxtral, Gemini TTS) before any paid one.

A weekly scout keeps the knowledge base current. Every Monday at 07:43 a scheduled job collects new repositories, Jev filters them, and Claude Opus 5.5 reads the shortlist and files what it finds. It may only edit the knowledge base and its scan reports; a wrapper checks that nothing else changed, commits, and shows a notification. Each report ends with at most three "checklist candidates", practices worth trying. A candidate joins the studio's checklist only after a real piece has tested it.

Two frames of the same clay scene, a tablet line with a hopper, a press, a belt and a blister pack, rendered twice: the top row by Blender Cycles at about 7.7 seconds per frame, the bottom row by Three.js in headless Chrome at about 0.05 seconds per frame.
The same clay scene from one cue file: Blender's light is softer, while Three.js renders each frame about 150 times faster.
A contact sheet of 17 numbered frames from the clay animation, from the hopper and press through the belt and blister pack to a closed box with a cross on its lid.
The model reviews its own film through contact sheets like this one, one frame for every second of film, and fixes what it sees.
A terminal-style report titled 'Final QA: powder-to-pill-v4.mp4' marked PASS, with rows for decode, loudness, silence and flash checks, all passing, and a pointer to the contact sheet.
One command checks the finished file for decoding, loudness, silences and flashes before anyone watches it.

Equipment

How I built it

  1. I started with a research report on how the first code-drawn films were made, then built the knowledge base around what it found: the quality comes less from the drawing library than from the brief the model follows (storyboard first, one event per shot, timed reads) and from a review loop in which the model looks at contact sheets of its own frames and fixes what it sees.
  2. Next I made a 15-second test cartoon with a public painted-animation kit (p5.js and p5.brush) and a soundtrack synthesised in code, and reviewed it in four rounds of contact sheets and frame strips.
  3. The scout runs on a schedule, with a cheap filter in front of it. Plain code collects candidates from GitHub searches, README searches and curated lists. Jev scores each README for relevance and substance and tags it; Opus judges quality only on the shortlist.
  4. A ledger measures every job from Claude Code's own transcripts: tokens, an API-equivalent cost, working time, render time and generation credits. The figures are a yardstick. I don't pay them, because the work runs on a subscription.
  5. Checks for finished videos had to prove they can fail before I trusted them: one command checks decoding, duration and frame count, loudness and true peak, silences, flashes (WCAG 2.3.1), speech-to-text against the script, and whether every on-screen text stays up long enough to read.
  6. Comparisons held everything else equal: the same clay scene in Blender and in Three.js from one cue file, the same brief to two motion tools, and the scout's readers on Opus 5.5 versus Sonnet 5.5.

Results

  • The 15-second test cartoon: 360 frames rendered on 4 workers in 99 s; about 13 minutes of assistant work from go-ahead to finished video, including the storyboard and 4 review rounds; 8.0 million tokens, about US$3.22 API-equivalent, 0 generation credits. My earlier guess had been about an hour.
  • Triage: 240 candidates scored in 36 s for US$0.018. Keeping only relevance 3 or higher cut the list to about 80 and lost none of the five repositories Opus had picked by hand. As a quality ranker Jev was weak (those five ranked 10, 14, 16, 23 and 60), so Jev filters and Opus judges.
  • The scout: eight scan reports from 2026-09-24 to 2026-10-19. The latest run took 13.6 minutes and about US$17.26 API-equivalent to go through 653 new repositories, shortlist 90 and read about 65 with five parallel readers.
  • Fixing the collector: replayed on an old week, it found 1,130 candidates instead of 590.
  • Readers: Sonnet 5.5 readers cost 32% less than Opus 5.5 readers on the same week (US$13.61 against US$20.05) and filed 39 repositories against 38, 24 in common. The default stays Opus until the effort question is settled.
  • Blender versus Three.js: 204 frames at 1600 by 900 took 26 to 27.5 minutes per render in Blender Cycles and about 10 s in Three.js, about 150 times faster. Story and animation scored the same; the clay look was better in Cycles, because a rasteriser has no subsurface scattering or bounce light.
  • Two tools, one brief: both builders finished in 16 to 19 minutes. I preferred one tool's look; the other matched it in one 10-minute round once it was given reference frames and five named qualities.

What I learned

Measure instead of guessing. The test cartoon took about 13 minutes of assistant work, not the hour I had estimated. Every job now gets a ledger row.

Automated jobs fail silently. GitHub's limit of 30 searches a minute had been killing every README search without an error, and a too-strict guardrail meant one weekly scan was never committed. Both went unnoticed until 2026-10-15. Now a missing scan sends me to the logs first.

A polished look comes from the visual language more than from the tool. Before switching tools to get a look, I rebuild against a reference frame, with the qualities I want named in a short list.

Lab notes

  1. I found that GitHub's search rate limit had been silently breaking the scout's collector and fixed it with paced, retried searches. Its eighth scan report followed: 653 new repositories, 90 shortlisted, about 65 read.

  2. I wrote a pre-flight checklist and a one-command check of the finished file, with every test first proven to fail on planted defects. Then I rendered the same clay scene in Blender and in Three.js from one cue file, and gave the same brief to two motion tools.

  3. I put a Jev triage step in front of Opus to filter new GitHub repositories, added a ledger that measures tokens and time for every job, and widened the knowledge base to generative and hybrid video.

  4. I researched how the first code-drawn films were being made, started the knowledge base, made a 15-second test cartoon and set up a scout that runs every Monday.

© 2026 Johnattan García-Ruiz Last updated