Out of Paper: from stills to film
The paper films never used a video model: everything moved in code. The natural next step was to add one. This time I wanted to blend very different design styles in one film, mix image and video models, and have Claude work with music that has lyrics. The method: generate a still for each shot, approve it, and only then ask a video model to animate it.
The film picks up where the sign video ended. My paper puppet sits on the sofa under the sign, then pushes through the wall, and on the other side are short scenes of things I love doing, each in a different style: photoreal climbing, anime sailing, pop-art cooking, minimalist chess, pixel-art snowboarding, a flat illustration of tinkering, then a flip-book of beach, reading, garden and friends. It ends with a rewind back to the sofa. The song, with lyrics, was generated too.
One person, seven styles.
The tools
Everything generated went through Opper, for convenience: one key and one bill for every model below, instead of an account with each provider.
- Stills: OpenAI’s gpt-image-2.5 (Sunburst), for every first frame, every last frame and the identity sheet.
- Video: Kling 3.0 Pro, image to video, given a first and a last frame per clip. In an earlier test against Seedance 2.5 and Gemini Omni it was the best at acting and at landing on the exact last frame, and the cheapest; here it also held every art style for the full five seconds.
- Music: ElevenLabs music, a song with written lyrics, one line per section.
- Transcription: ElevenLabs Scribe, for the timing of every sung word (the subtitles).
- Listening checks: Gemini, asked to listen to the mix.
- Putting it together: Claude Code wrote the prompts, the cue file and the composition; HyperFrames rendered it, as in part 1, and FFmpeg mixed and mastered.
Before starting
I read a published making-of of a three-minute AI music video and wrote down what they had learned. The most useful lesson: write each clip prompt as a timed script, not a description. “He cooks” gives a lifeless take. A paragraph of “0.0 to 0.6 s: …, 0.63 s: the flame jumps, …” with the sounds listed in order gives a scene.
What I learned
- Never put the real photos in the film. My photos became one identity sheet, and only the sheet was used as a reference. My likeness held in every style.
- Strong styles fight a photo reference. Claymation came back photoreal twice. Generating with the sheet first and then restyling that result on its own fixed it.
- Ask for the last frame too. Each clip got its first frame and an edited last frame (eyes closed, stove off, mid-jump), so it ended exactly where the next transition needed it.
- Transitions need a reason. A blink, a freeze into halftone dots, the stove going dark and a lamp flickering on, a pixel dissolve into the pixel-art scene, brush strokes, page turns. Each one starts from something happening at the end of a shot and lands on a matching shape in the next.
- Retakes are cheap, so take them. The cooking scene came back once with a fireball and once with a cloud of salt over my face. “Nothing in front of his face” fixed the third take.
- Transcribe the song. One of three takes sang invented words. The subtitled version uses word timings from a transcript, with the written lyrics on screen.
What it cost
| Cost | |
|---|---|
| Video clips (eight takes, one model, sound off) | $4.48 |
| Stills | $1.21 |
| Song | $0.43 |
| Total | about $6.10 |