docs: add bilingual README (English + Chinese)
This commit is contained in:
@@ -0,0 +1,92 @@
|
||||
# Video Edit Planner
|
||||
|
||||
An agent skill that helps plan video edits through iterative dialogue — transcribe audio, extract key frames on demand, analyze visuals, and produce a structured edit plan.
|
||||
|
||||
**Repository**: https://git.nite07.com/nite/video-edit-planner-skill
|
||||
|
||||
## Features
|
||||
|
||||
- **Audio transcription** — bundled [funasr-script](https://github.com/modelscope/FunASR) with Fun-ASR-Nano (high quality, Chinese-optimized) and SenseVoice (fast preview) models
|
||||
- **On-demand frame extraction** — ffmpeg-based clip cutting (`-c copy`) + scene-change detection + uniform sampling, with hardware acceleration (CUDA/NVDEC preferred)
|
||||
- **Artifact indexing** — SQLite database tracks all transcriptions, clips, and frames to avoid duplicate processing across sessions
|
||||
- **Vision analysis guidance** — binary-search-style frame sampling strategy; works with any vision-capable model the agent's runtime provides
|
||||
- **Iterative edit plans** — Markdown-table output (timecodes, segment descriptions, actions, transitions, notes) refined through follow-up questions
|
||||
- **Agent-agnostic** — no platform-specific tool names; works with any agent framework (Hermes, Claude Code, Codex, etc.)
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
| Dependency | Install |
|
||||
|---|---|
|
||||
| `ffmpeg` + `ffprobe` | Linux: `pacman -S ffmpeg` / `apt install ffmpeg`; macOS: `brew install ffmpeg`; Windows: `winget install Gyan.FFmpeg` |
|
||||
| `uv` | Linux: `pacman -S uv`; macOS: `brew install uv`; Windows: `winget install astral-sh.uv`; fallback: `pip install uv` |
|
||||
| `python3` | Linux: `pacman -S python`; macOS: `brew install python`; Windows: `winget install Python.Python.3` |
|
||||
|
||||
### Install the skill
|
||||
|
||||
Copy or clone this repository into your agent's skills directory. For Hermes Agent:
|
||||
|
||||
```bash
|
||||
git clone mygit:nite/video-edit-planner-skill.git ~/.hermes/skills/media/video-edit-planner
|
||||
```
|
||||
|
||||
### First use
|
||||
|
||||
The transcription scripts need a one-time dependency setup:
|
||||
|
||||
```bash
|
||||
cd ~/.hermes/skills/media/video-edit-planner/scripts/transcription && uv sync
|
||||
```
|
||||
|
||||
This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If you've used these packages via uv before, the cache is reused — first-time cost is only the link step.
|
||||
|
||||
## Workflow
|
||||
|
||||
```
|
||||
1. Check dependencies (ffmpeg, uv, python3)
|
||||
2. Gather inputs (video path + audio track index)
|
||||
3. Ask editing requirements
|
||||
4. Transcribe audio (skip if cached in index)
|
||||
5. Extract clips & frames on demand (agent decides when transcript is insufficient)
|
||||
6. Analyze frames with vision model (binary-search-style sampling)
|
||||
7. Produce Markdown-table edit plan + overall recommendation
|
||||
8. Iterate — user asks follow-ups, plan is refined
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
video-edit-planner/
|
||||
├── SKILL.md # Skill definition (workflow, guidance, pitfalls)
|
||||
├── README.md # This file (English)
|
||||
├── README.zh.md # 中文说明
|
||||
├── scripts/
|
||||
│ ├── transcription/ # Bundled funasr-script (self-contained uv project)
|
||||
│ │ ├── pyproject.toml
|
||||
│ │ ├── uv.lock
|
||||
│ │ ├── funasr_common.py # Shared: ffprobe, audio extraction, model runner
|
||||
│ │ ├── funasr_nano.py # Fun-ASR-Nano entry point (high quality)
|
||||
│ │ ├── funasr_fast.py # SenseVoice entry point (fast preview)
|
||||
│ │ └── funasr_regular.py # Paraformer entry point (comparison)
|
||||
│ ├── frames/
|
||||
│ │ └── extract_frames.py # Clip extraction + frame sampling (ffmpeg wrapper)
|
||||
│ └── index/
|
||||
│ └── manage_index.py # SQLite index management (8 subcommands)
|
||||
└── references/
|
||||
└── frame-extraction-guide.md # Vision model token costs, resolution/batch guidance
|
||||
```
|
||||
|
||||
## Index Database
|
||||
|
||||
All processing artifacts are tracked in an SQLite database (`<video_stem>.vedit.db`) stored next to the video file:
|
||||
|
||||
- **transcription** — JSON path, track index, duration
|
||||
- **clips** — start/end time, file path, extraction reason
|
||||
- **frames** — timestamp, file path, scene score, extraction method
|
||||
|
||||
This avoids re-transcribing or re-extracting frames for the same video across sessions.
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
@@ -0,0 +1,92 @@
|
||||
# Video Edit Planner 视频剪辑规划助手
|
||||
|
||||
一个 agent skill，通过迭代式对话帮助规划视频剪辑——转录音频、按需提取关键帧、分析画面内容，最终生成结构化的剪辑规划。
|
||||
|
||||
**仓库地址**: https://git.nite07.com/nite/video-edit-planner-skill
|
||||
|
||||
## 功能特性
|
||||
|
||||
- **音频转录** — 内置 [funasr-script](https://github.com/modelscope/FunASR)，支持 Fun-ASR-Nano（高质量，中文优化）和 SenseVoice（快速预览）两种模型
|
||||
- **按需抽帧** — 基于 ffmpeg 的片段剪辑（`-c copy`）+ 场景变化检测 + 均匀采样，优先使用硬件加速（CUDA/NVDEC）
|
||||
- **产物索引** — SQLite 数据库记录所有转录、剪辑片段和帧图片，避免跨会话重复处理
|
||||
- **视觉分析指引** — 二分查找式帧采样策略；兼容 agent 运行时提供的任何视觉模型
|
||||
- **迭代式剪辑规划** — Markdown 表格输出（时间码、片段描述、操作建议、转场方式、备注），支持追问细化
|
||||
- **Agent 无关** — 不包含任何平台特定工具名；兼容任何 agent 框架（Hermes、Claude Code、Codex 等）
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 前置依赖
|
||||
|
||||
| 依赖 | 安装方式 |
|
||||
|---|---|
|
||||
| `ffmpeg` + `ffprobe` | Linux: `pacman -S ffmpeg` / `apt install ffmpeg`；macOS: `brew install ffmpeg`；Windows: `winget install Gyan.FFmpeg` |
|
||||
| `uv` | Linux: `pacman -S uv`；macOS: `brew install uv`；Windows: `winget install astral-sh.uv`；备用: `pip install uv` |
|
||||
| `python3` | Linux: `pacman -S python`；macOS: `brew install python`；Windows: `winget install Python.Python.3` |
|
||||
|
||||
### 安装 skill
|
||||
|
||||
将本仓库克隆到你的 agent 的 skills 目录。以 Hermes Agent 为例：
|
||||
|
||||
```bash
|
||||
git clone mygit:nite/video-edit-planner-skill.git ~/.hermes/skills/media/video-edit-planner
|
||||
```
|
||||
|
||||
### 首次使用
|
||||
|
||||
转录脚本需要一次性依赖安装：
|
||||
|
||||
```bash
|
||||
cd ~/.hermes/skills/media/video-edit-planner/scripts/transcription && uv sync
|
||||
```
|
||||
|
||||
这会创建一个隔离的 venv，包含 `funasr`、`torch`、`torchaudio`。如果你之前通过 uv 安装过这些包，缓存会被复用——首次成本仅为链接步骤。
|
||||
|
||||
## 工作流程
|
||||
|
||||
```
|
||||
1. 检查依赖（ffmpeg、uv、python3）
|
||||
2. 收集输入（视频路径 + 音轨索引）
|
||||
3. 询问剪辑需求
|
||||
4. 转录音频（如索引中已有缓存则跳过）
|
||||
5. 按需提取片段和帧（agent 自主判断转录稿是否足够）
|
||||
6. 用视觉模型分析帧（二分查找式采样）
|
||||
7. 生成 Markdown 表格剪辑规划 + 总体建议
|
||||
8. 迭代——用户追问，规划逐步细化
|
||||
```
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
video-edit-planner/
|
||||
├── SKILL.md # Skill 定义（工作流程、指引、注意事项）
|
||||
├── README.md # 英文说明
|
||||
├── README.zh.md # 本文件（中文）
|
||||
├── scripts/
|
||||
│ ├── transcription/ # 内置 funasr-script（自包含 uv 项目）
|
||||
│ │ ├── pyproject.toml
|
||||
│ │ ├── uv.lock
|
||||
│ │ ├── funasr_common.py # 共享：ffprobe、音频提取、模型运行
|
||||
│ │ ├── funasr_nano.py # Fun-ASR-Nano 入口（高质量）
|
||||
│ │ ├── funasr_fast.py # SenseVoice 入口（快速预览）
|
||||
│ │ └── funasr_regular.py # Paraformer 入口（对比用）
|
||||
│ ├── frames/
|
||||
│ │ └── extract_frames.py # 片段提取 + 抽帧（ffmpeg 封装）
|
||||
│ └── index/
|
||||
│ └── manage_index.py # SQLite 索引管理（8 个子命令）
|
||||
└── references/
|
||||
└── frame-extraction-guide.md # 视觉模型 token 成本、分辨率/批次指引
|
||||
```
|
||||
|
||||
## 索引数据库
|
||||
|
||||
所有处理产物记录在 SQLite 数据库（`<视频文件名>.vedit.db`）中，存放在视频文件同目录：
|
||||
|
||||
- **transcription** — JSON 路径、音轨索引、时长
|
||||
- **clips** — 起止时间、文件路径、提取原因
|
||||
- **frames** — 时间戳、文件路径、场景分数、提取方法
|
||||
|
||||
这避免了跨会话对同一视频重复转录或重复抽帧。
|
||||
|
||||
## 许可证
|
||||
|
||||
MIT
|
||||
Reference in New Issue
Block a user