nite
5f9e96de19
README (EN + ZH): - npx skills add: use HTTPS URL instead of SSH - Remove 'First use' uv sync section (agent handles setup, not user) - Remove 'Stock Material Search' section (workflow detail, belongs in SKILL.md) - Keep README focused on overview + workflow, not implementation details - Fun-ASR-Nano marked as default in Features SKILL.md: - Step 4: explicitly state Fun-ASR-Nano as recommended default model
101 lines
4.6 KiB
Markdown
101 lines
4.6 KiB
Markdown
# Video Edit Planner
|
|
|
|
An agent skill that helps plan video edits through iterative dialogue — transcribe audio, extract key frames on demand, analyze visuals, and produce a structured edit plan.
|
|
|
|
**Repository**: https://git.nite07.com/nite/video-edit-planner-skill
|
|
|
|
## Features
|
|
|
|
- **Audio transcription** — bundled [funasr-script](https://github.com/modelscope/FunASR) with Fun-ASR-Nano (default, high quality, Chinese-optimized) and SenseVoice (fast preview) models
|
|
- **On-demand frame extraction** — ffmpeg-based clip cutting (`-c copy`) + scene-change detection + uniform sampling, with hardware acceleration (CUDA/NVDEC preferred)
|
|
- **Artifact indexing** — SQLite database tracks all transcriptions, clips, and frames to avoid duplicate processing across sessions
|
|
- **Vision analysis guidance** — binary-search-style frame sampling strategy; works with any vision-capable model the agent's runtime provides
|
|
- **Iterative edit plans** — Markdown-table output (timecodes, segment descriptions, actions, transitions, notes) refined through follow-up questions
|
|
- **Agent-agnostic** — no platform-specific tool names; works with any agent framework (Hermes, Claude Code, Codex, etc.)
|
|
|
|
## Quick Start
|
|
|
|
### Prerequisites
|
|
|
|
| Dependency | Install |
|
|
|---|---|
|
|
| `ffmpeg` + `ffprobe` | Linux: `pacman -S ffmpeg` / `apt install ffmpeg`; macOS: `brew install ffmpeg`; Windows: `winget install Gyan.FFmpeg` |
|
|
| `uv` | Linux: `pacman -S uv`; macOS: `brew install uv`; Windows: `winget install astral-sh.uv`; fallback: `pip install uv` |
|
|
| `python3` | Linux: `pacman -S python`; macOS: `brew install python`; Windows: `winget install Python.Python.3` |
|
|
|
|
### Install the skill
|
|
|
|
**Option 1: `npx skills add` (recommended)**
|
|
|
|
This skill is compatible with the [open agent skills ecosystem](https://www.npmjs.com/package/skills). You can install it directly:
|
|
|
|
```bash
|
|
# Install globally (available across all projects)
|
|
npx skills add https://git.nite07.com/nite/video-edit-planner-skill.git -g -y
|
|
|
|
# Install to specific agents
|
|
npx skills add https://git.nite07.com/nite/video-edit-planner-skill.git -g -a claude-code -y
|
|
|
|
# List available skills without installing
|
|
npx skills add https://git.nite07.com/nite/video-edit-planner-skill.git --list
|
|
```
|
|
|
|
Supports 73+ agent frameworks including Claude Code, Codex, Cursor, OpenCode, and more.
|
|
|
|
**Option 2: `git clone` (manual)**
|
|
|
|
```bash
|
|
git clone https://git.nite07.com/nite/video-edit-planner-skill.git ~/.hermes/skills/media/video-edit-planner
|
|
```
|
|
|
|
For Hermes Agent specifically, clone into `~/.hermes/skills/media/`. For other agents, follow their respective skills directory conventions.
|
|
|
|
## Workflow
|
|
|
|
```
|
|
1. Check dependencies (ffmpeg, uv, python3)
|
|
2. Gather inputs (video path + audio track index)
|
|
3. Ask editing requirements
|
|
4. Transcribe audio (skip if cached in index)
|
|
5. Extract clips & frames on demand (agent decides when transcript is insufficient)
|
|
6. Analyze frames with vision model (binary-search-style sampling)
|
|
7. Produce Markdown-table edit plan + overall recommendation
|
|
8. Iterate — user asks follow-ups, plan is refined
|
|
```
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
video-edit-planner/
|
|
├── SKILL.md # Skill definition (workflow, guidance, pitfalls)
|
|
├── README.md # This file (English)
|
|
├── README.zh.md # 中文说明
|
|
├── scripts/
|
|
│ ├── transcription/ # Bundled funasr-script (self-contained uv project)
|
|
│ │ ├── pyproject.toml
|
|
│ │ ├── uv.lock
|
|
│ │ ├── funasr_common.py # Shared: ffprobe, audio extraction, model runner
|
|
│ │ ├── funasr_nano.py # Fun-ASR-Nano entry point (default, high quality)
|
|
│ │ ├── funasr_fast.py # SenseVoice entry point (fast preview)
|
|
│ │ └── funasr_regular.py # Paraformer entry point (comparison)
|
|
│ ├── frames/
|
|
│ │ └── extract_frames.py # Clip extraction + frame sampling (ffmpeg wrapper)
|
|
│ └── index/
|
|
│ └── manage_index.py # SQLite index management (8 subcommands)
|
|
└── references/
|
|
└── frame-extraction-guide.md # Vision model token costs, resolution/batch guidance
|
|
```
|
|
|
|
## Index Database
|
|
|
|
All processing artifacts are tracked in an SQLite database (`<video_stem>.vedit.db`) stored next to the video file:
|
|
|
|
- **transcription** — JSON path, track index, duration
|
|
- **clips** — start/end time, file path, extraction reason
|
|
- **frames** — timestamp, file path, scene score, extraction method
|
|
|
|
This avoids re-transcribing or re-extracting frames for the same video across sessions.
|
|
|
|
## License
|
|
|
|
MIT |