# Video Edit Planner

An agent skill that helps plan video edits through iterative dialogue — transcribe audio, extract key frames on demand, analyze visuals, and produce a structured edit plan.

**Repository**: https://git.nite07.com/nite/video-edit-planner-skill

## Features

- **Audio transcription** — bundled [funasr-script](https://github.com/modelscope/FunASR) with Fun-ASR-Nano (high quality, Chinese-optimized) and SenseVoice (fast preview) models
- **On-demand frame extraction** — ffmpeg-based clip cutting (`-c copy`) + scene-change detection + uniform sampling, with hardware acceleration (CUDA/NVDEC preferred)
- **Artifact indexing** — SQLite database tracks all transcriptions, clips, and frames to avoid duplicate processing across sessions
- **Vision analysis guidance** — binary-search-style frame sampling strategy; works with any vision-capable model the agent's runtime provides
- **Iterative edit plans** — Markdown-table output (timecodes, segment descriptions, actions, transitions, notes) refined through follow-up questions
- **Agent-agnostic** — no platform-specific tool names; works with any agent framework (Hermes, Claude Code, Codex, etc.)

## Quick Start

### Prerequisites

| Dependency | Install |
|---|---|
| `ffmpeg` + `ffprobe` | Linux: `pacman -S ffmpeg` / `apt install ffmpeg`; macOS: `brew install ffmpeg`; Windows: `winget install Gyan.FFmpeg` |
| `uv` | Linux: `pacman -S uv`; macOS: `brew install uv`; Windows: `winget install astral-sh.uv`; fallback: `pip install uv` |
| `python3` | Linux: `pacman -S python`; macOS: `brew install python`; Windows: `winget install Python.Python.3` |

### Install the skill

**Option 1: `npx skills add` (recommended)**

This skill is compatible with the [open agent skills ecosystem](https://www.npmjs.com/package/skills). You can install it directly:

```bash
# Install globally (available across all projects)
npx skills add git@git.nite07.com:nite/video-edit-planner-skill.git -g -y

# Install to specific agents
npx skills add git@git.nite07.com:nite/video-edit-planner-skill.git -g -a claude-code -y

# List available skills without installing
npx skills add git@git.nite07.com:nite/video-edit-planner-skill.git --list
```

Supports 73+ agent frameworks including Claude Code, Codex, Cursor, OpenCode, and more.

**Option 2: `git clone` (manual)**

```bash
git clone git@git.nite07.com:nite/video-edit-planner-skill.git ~/.hermes/skills/media/video-edit-planner
```

For Hermes Agent specifically, clone into `~/.hermes/skills/media/`. For other agents, follow their respective skills directory conventions.

### First use

The transcription scripts need a one-time dependency setup:

```bash
cd ~/.hermes/skills/media/video-edit-planner/scripts/transcription && uv sync
```

This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If you've used these packages via uv before, the cache is reused — first-time cost is only the link step.

## Workflow

```
1. Check dependencies (ffmpeg, uv, python3)
2. Gather inputs (video path + audio track index)
3. Ask editing requirements
4. Transcribe audio (skip if cached in index)
5. Extract clips & frames on demand (agent decides when transcript is insufficient)
6. Analyze frames with vision model (binary-search-style sampling)
7. Produce Markdown-table edit plan + overall recommendation
8. Iterate — user asks follow-ups, plan is refined
```

## Stock Material Search (optional)

When planning edits, users may need stock materials — stickers, GIFs, meme video clips, or B-roll. The skill provides guidance for finding these resources on demand.

### Material sources

| Source | URL | Content | Best for |
|---|---|---|---|
| 爱给网 | https://www.aigei.com/ | Chinese meme video clips (热梗, 42K+), free download | Chinese-language creators (B站/抖音) |
| B站素材酷 | https://cool.bilibili.com/ | Bilibili platform-native assets (video, audio, BGM, templates, stickers) | B站 creators; large Chinese meme collection |
| Tenor | https://tenor.com/ | GIFs + stickers, 30+ languages incl. Chinese, free API | Quick GIF/sticker search |
| GIPHY | https://giphy.com/ | Largest GIF library, 30+ languages, sticker channel | English/multilingual GIF search |
| Klipy | https://klipy.com/ | Localized GIF/sticker/clip API, supports Chinese | API-integrated search, localization |

For Chinese-language users, 爱给网 and B站素材酷 are recommended first.

### B站素材酷 search

B站素材酷 has no built-in search. Use the [bcut-resource-search](https://git.nite07.com/nite/bcut-resource-search) Tampermonkey userscript to add search functionality, or let the agent query the API directly.

## Project Structure

```
video-edit-planner/
├── SKILL.md                          # Skill definition (workflow, guidance, pitfalls)
├── README.md                         # This file (English)
├── README.zh.md                      # 中文说明
├── scripts/
│   ├── transcription/                # Bundled funasr-script (self-contained uv project)
│   │   ├── pyproject.toml
│   │   ├── uv.lock
│   │   ├── funasr_common.py          # Shared: ffprobe, audio extraction, model runner
│   │   ├── funasr_nano.py            # Fun-ASR-Nano entry point (high quality)
│   │   ├── funasr_fast.py            # SenseVoice entry point (fast preview)
│   │   └── funasr_regular.py         # Paraformer entry point (comparison)
│   ├── frames/
│   │   └── extract_frames.py         # Clip extraction + frame sampling (ffmpeg wrapper)
│   └── index/
│       └── manage_index.py           # SQLite index management (8 subcommands)
└── references/
    └── frame-extraction-guide.md     # Vision model token costs, resolution/batch guidance
```

## Index Database

All processing artifacts are tracked in an SQLite database (`<video_stem>.vedit.db`) stored next to the video file:

- **transcription** — JSON path, track index, duration
- **clips** — start/end time, file path, extraction reason
- **frames** — timestamp, file path, scene score, extraction method

This avoids re-transcribing or re-extracting frames for the same video across sessions.

## License

MIT