From f8ecbadf1d39c90efa407e8da35e0975f1fbaf29 Mon Sep 17 00:00:00 2001 From: nite Date: Tue, 14 Jul 2026 11:09:15 +0800 Subject: [PATCH] docs: add bilingual README (English + Chinese) --- README.md | 92 ++++++++++++++++++++++++++++++++++++++++++++++++++++ README.zh.md | 92 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 184 insertions(+) create mode 100644 README.md create mode 100644 README.zh.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..1c111ff --- /dev/null +++ b/README.md @@ -0,0 +1,92 @@ +# Video Edit Planner + +An agent skill that helps plan video edits through iterative dialogue — transcribe audio, extract key frames on demand, analyze visuals, and produce a structured edit plan. + +**Repository**: https://git.nite07.com/nite/video-edit-planner-skill + +## Features + +- **Audio transcription** — bundled [funasr-script](https://github.com/modelscope/FunASR) with Fun-ASR-Nano (high quality, Chinese-optimized) and SenseVoice (fast preview) models +- **On-demand frame extraction** — ffmpeg-based clip cutting (`-c copy`) + scene-change detection + uniform sampling, with hardware acceleration (CUDA/NVDEC preferred) +- **Artifact indexing** — SQLite database tracks all transcriptions, clips, and frames to avoid duplicate processing across sessions +- **Vision analysis guidance** — binary-search-style frame sampling strategy; works with any vision-capable model the agent's runtime provides +- **Iterative edit plans** — Markdown-table output (timecodes, segment descriptions, actions, transitions, notes) refined through follow-up questions +- **Agent-agnostic** — no platform-specific tool names; works with any agent framework (Hermes, Claude Code, Codex, etc.) + +## Quick Start + +### Prerequisites + +| Dependency | Install | +|---|---| +| `ffmpeg` + `ffprobe` | Linux: `pacman -S ffmpeg` / `apt install ffmpeg`; macOS: `brew install ffmpeg`; Windows: `winget install Gyan.FFmpeg` | +| `uv` | Linux: `pacman -S uv`; macOS: `brew install uv`; Windows: `winget install astral-sh.uv`; fallback: `pip install uv` | +| `python3` | Linux: `pacman -S python`; macOS: `brew install python`; Windows: `winget install Python.Python.3` | + +### Install the skill + +Copy or clone this repository into your agent's skills directory. For Hermes Agent: + +```bash +git clone mygit:nite/video-edit-planner-skill.git ~/.hermes/skills/media/video-edit-planner +``` + +### First use + +The transcription scripts need a one-time dependency setup: + +```bash +cd ~/.hermes/skills/media/video-edit-planner/scripts/transcription && uv sync +``` + +This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If you've used these packages via uv before, the cache is reused — first-time cost is only the link step. + +## Workflow + +``` +1. Check dependencies (ffmpeg, uv, python3) +2. Gather inputs (video path + audio track index) +3. Ask editing requirements +4. Transcribe audio (skip if cached in index) +5. Extract clips & frames on demand (agent decides when transcript is insufficient) +6. Analyze frames with vision model (binary-search-style sampling) +7. Produce Markdown-table edit plan + overall recommendation +8. Iterate — user asks follow-ups, plan is refined +``` + +## Project Structure + +``` +video-edit-planner/ +├── SKILL.md # Skill definition (workflow, guidance, pitfalls) +├── README.md # This file (English) +├── README.zh.md # 中文说明 +├── scripts/ +│ ├── transcription/ # Bundled funasr-script (self-contained uv project) +│ │ ├── pyproject.toml +│ │ ├── uv.lock +│ │ ├── funasr_common.py # Shared: ffprobe, audio extraction, model runner +│ │ ├── funasr_nano.py # Fun-ASR-Nano entry point (high quality) +│ │ ├── funasr_fast.py # SenseVoice entry point (fast preview) +│ │ └── funasr_regular.py # Paraformer entry point (comparison) +│ ├── frames/ +│ │ └── extract_frames.py # Clip extraction + frame sampling (ffmpeg wrapper) +│ └── index/ +│ └── manage_index.py # SQLite index management (8 subcommands) +└── references/ + └── frame-extraction-guide.md # Vision model token costs, resolution/batch guidance +``` + +## Index Database + +All processing artifacts are tracked in an SQLite database (`.vedit.db`) stored next to the video file: + +- **transcription** — JSON path, track index, duration +- **clips** — start/end time, file path, extraction reason +- **frames** — timestamp, file path, scene score, extraction method + +This avoids re-transcribing or re-extracting frames for the same video across sessions. + +## License + +MIT \ No newline at end of file diff --git a/README.zh.md b/README.zh.md new file mode 100644 index 0000000..ec77c32 --- /dev/null +++ b/README.zh.md @@ -0,0 +1,92 @@ +# Video Edit Planner 视频剪辑规划助手 + +一个 agent skill,通过迭代式对话帮助规划视频剪辑——转录音频、按需提取关键帧、分析画面内容,最终生成结构化的剪辑规划。 + +**仓库地址**: https://git.nite07.com/nite/video-edit-planner-skill + +## 功能特性 + +- **音频转录** — 内置 [funasr-script](https://github.com/modelscope/FunASR),支持 Fun-ASR-Nano(高质量,中文优化)和 SenseVoice(快速预览)两种模型 +- **按需抽帧** — 基于 ffmpeg 的片段剪辑(`-c copy`)+ 场景变化检测 + 均匀采样,优先使用硬件加速(CUDA/NVDEC) +- **产物索引** — SQLite 数据库记录所有转录、剪辑片段和帧图片,避免跨会话重复处理 +- **视觉分析指引** — 二分查找式帧采样策略;兼容 agent 运行时提供的任何视觉模型 +- **迭代式剪辑规划** — Markdown 表格输出(时间码、片段描述、操作建议、转场方式、备注),支持追问细化 +- **Agent 无关** — 不包含任何平台特定工具名;兼容任何 agent 框架(Hermes、Claude Code、Codex 等) + +## 快速开始 + +### 前置依赖 + +| 依赖 | 安装方式 | +|---|---| +| `ffmpeg` + `ffprobe` | Linux: `pacman -S ffmpeg` / `apt install ffmpeg`;macOS: `brew install ffmpeg`;Windows: `winget install Gyan.FFmpeg` | +| `uv` | Linux: `pacman -S uv`;macOS: `brew install uv`;Windows: `winget install astral-sh.uv`;备用: `pip install uv` | +| `python3` | Linux: `pacman -S python`;macOS: `brew install python`;Windows: `winget install Python.Python.3` | + +### 安装 skill + +将本仓库克隆到你的 agent 的 skills 目录。以 Hermes Agent 为例: + +```bash +git clone mygit:nite/video-edit-planner-skill.git ~/.hermes/skills/media/video-edit-planner +``` + +### 首次使用 + +转录脚本需要一次性依赖安装: + +```bash +cd ~/.hermes/skills/media/video-edit-planner/scripts/transcription && uv sync +``` + +这会创建一个隔离的 venv,包含 `funasr`、`torch`、`torchaudio`。如果你之前通过 uv 安装过这些包,缓存会被复用——首次成本仅为链接步骤。 + +## 工作流程 + +``` +1. 检查依赖(ffmpeg、uv、python3) +2. 收集输入(视频路径 + 音轨索引) +3. 询问剪辑需求 +4. 转录音频(如索引中已有缓存则跳过) +5. 按需提取片段和帧(agent 自主判断转录稿是否足够) +6. 用视觉模型分析帧(二分查找式采样) +7. 生成 Markdown 表格剪辑规划 + 总体建议 +8. 迭代——用户追问,规划逐步细化 +``` + +## 目录结构 + +``` +video-edit-planner/ +├── SKILL.md # Skill 定义(工作流程、指引、注意事项) +├── README.md # 英文说明 +├── README.zh.md # 本文件(中文) +├── scripts/ +│ ├── transcription/ # 内置 funasr-script(自包含 uv 项目) +│ │ ├── pyproject.toml +│ │ ├── uv.lock +│ │ ├── funasr_common.py # 共享:ffprobe、音频提取、模型运行 +│ │ ├── funasr_nano.py # Fun-ASR-Nano 入口(高质量) +│ │ ├── funasr_fast.py # SenseVoice 入口(快速预览) +│ │ └── funasr_regular.py # Paraformer 入口(对比用) +│ ├── frames/ +│ │ └── extract_frames.py # 片段提取 + 抽帧(ffmpeg 封装) +│ └── index/ +│ └── manage_index.py # SQLite 索引管理(8 个子命令) +└── references/ + └── frame-extraction-guide.md # 视觉模型 token 成本、分辨率/批次指引 +``` + +## 索引数据库 + +所有处理产物记录在 SQLite 数据库(`<视频文件名>.vedit.db`)中,存放在视频文件同目录: + +- **transcription** — JSON 路径、音轨索引、时长 +- **clips** — 起止时间、文件路径、提取原因 +- **frames** — 时间戳、文件路径、场景分数、提取方法 + +这避免了跨会话对同一视频重复转录或重复抽帧。 + +## 许可证 + +MIT \ No newline at end of file