Pathwise AcademyOnline & In-Person AI ClassesIdea to BuildCreators Club · Ages 10–17Marketing with AI · AdultsCorporate WorkshopsHackathons for SchoolsSingaporeHong Kong
Pathwise AcademyAI classes for kids & adults
Find a class →Log in
← All free guides
Creating contentIntermediate · about 20 minFree · no signup

Break down a video frame by frame with Claude

yt-dlp downloads the video, FFmpeg pulls out the frames, and Claude reads what's on screen alongside what's said. Here's a prompt that runs the whole breakdown for you.

By Alan Brown · Works with Claude Code

Jump to the prompt ↓

👀 Here's how to let Claude actually watch your videos.

  • 📝 Give Claude a video link and it usually works from the transcript, so it misses everything happening on screen. Two free tools fix that.
  • ⬇️ yt-dlp downloads the video. github.com/yt-dlp/yt-dlp
  • 🎞️ FFmpeg pulls out the frames, so Claude can analyse it frame by frame alongside what's being said. ffmpeg.org
  • 🎬 Use it to break down reels, Shorts, YouTube videos or even a 30-minute recording.
  • 💻 It all runs locally on your computer, it's fast, and there are no extra API fees for the downloading and frame extraction.
  • 🛠️ How to install both, and a prompt for any FFmpeg job: The two tools that let Claude watch a video

Context & limitations

The downloading and frame extraction are free, but Claude still reads every frame you give it, so a long video at a high frame rate uses up your usage quickly. Start with a frame every few seconds. And yt-dlp doesn't know whose video it is: use it freely on your own uploads, but most platforms' terms don't allow downloading other people's.

Do this yourself

Help me break down a video frame by frame. The video is ___ (a link or a file path), and I want to know ___ (for example: why the hook works, how it's edited, or what's shown on screen). First check that yt-dlp and ffmpeg are installed (yt-dlp --version, ffmpeg -version). If either is missing, tell me the one install command for my system and stop. Then: 1. If it's a link, download it with yt-dlp into a new folder. If it's a file, use it where it is and don't change the original. 2. Use ffmpeg to save one frame every 2 seconds (every 10 seconds if it's longer than 5 minutes) and a separate audio file. 3. Get the transcript: use the video's own subtitles if yt-dlp can fetch them, otherwise tell me you're working without one. 4. Look at the frames in order, alongside the transcript. Give me: - A timeline table: timestamp, what's on screen, what's being said. - The on-screen text, cuts and visual tricks the transcript alone would miss. - Answers to my question, pointing to the timestamps that back each point. Only describe what you can actually see in the frames. If a frame is unclear, say so rather than guessing.

Claude Code can run yt-dlp and FFmpeg on your computer and then look at the frames itself. In the Claude app, run the two commands yourself and upload the frames instead.

Want to build this with a teacher?

Pathwise Academy runs small, hands-on AI classes in Singapore for ages 10–17 and adults.