From 7a4d3fc74bd439b9cddeddcf6810c194de424b07 Mon Sep 17 00:00:00 2001 From: nite Date: Wed, 15 Jul 2026 15:25:56 +0800 Subject: [PATCH] feat: subtractive editing as default strategy --- README.md | 3 +- README.zh.md | 3 +- SKILL.md | 83 ++++++++++++------- .../editing-requirements-questionnaire.md | 33 +++++--- 4 files changed, 77 insertions(+), 45 deletions(-) diff --git a/README.md b/README.md index b83320a..2eac140 100644 --- a/README.md +++ b/README.md @@ -11,6 +11,7 @@ An agent skill that helps plan video edits through iterative dialogue — transc - **Artifact indexing** — JSON file tracks all transcriptions, clips, and frames with relative paths to avoid duplicate processing across sessions - **Vision analysis guidance** — binary-search-style frame sampling strategy; works with any vision-capable model the agent's runtime provides - **Iterative edit plans** — Markdown-table output (timecodes, segment descriptions, actions, transitions, notes) refined through follow-up questions +- **Subtractive-first editing** — defaults to delete-boring-parts strategy for coherent timelines; supports additive and hybrid modes for Shorts/highlight reels - **Agent-agnostic** — no platform-specific tool names; works with any agent framework (Hermes, Claude Code, Codex, etc.) ## Quick Start @@ -55,7 +56,7 @@ For Hermes Agent specifically, clone into `~/.hermes/skills/media/`. For other a ``` 1. Check dependencies (ffmpeg, uv, python3) 2. Gather inputs (video path + audio track index) -3. Ask editing requirements +3. Ask editing requirements (including subtractive vs additive strategy) 4. Transcribe audio (skip if cached in index) 5. Extract clips & frames on demand (agent decides when transcript is insufficient) 6. Analyze frames with vision model (binary-search-style sampling) diff --git a/README.zh.md b/README.zh.md index 1c5c832..aaa3f5f 100644 --- a/README.zh.md +++ b/README.zh.md @@ -11,6 +11,7 @@ - **产物索引** — JSON 文件记录所有转录、剪辑片段和帧图片，使用相对路径，避免跨会话重复处理 - **视觉分析指引** — 二分查找式帧采样策略；兼容 agent 运行时提供的任何视觉模型 - **迭代式剪辑规划** — Markdown 表格输出（时间码、片段描述、操作建议、转场方式、备注），支持追问细化 +- **删去法优先** — 默认采用删去法策略（从完整时间线删除无聊段落），保证成片连贯性；同时支持拼接法和混合模式，适用于短视频/高光集锦 - **Agent 无关** — 不包含任何平台特定工具名；兼容任何 agent 框架（Hermes、Claude Code、Codex 等） ## 快速开始 @@ -55,7 +56,7 @@ Hermes Agent 用户克隆到 `~/.hermes/skills/media/`。其他 agent 请遵循 ``` 1. 检查依赖（ffmpeg、uv、python3） 2. 收集输入（视频路径 + 音轨索引） -3. 询问剪辑需求 +3. 询问剪辑需求（包括删去法 vs 拼接法策略选择） 4. 转录音频（如索引中已有缓存则跳过） 5. 按需提取片段和帧（agent 自主判断转录稿是否足够） 6. 用视觉模型分析帧（二分查找式采样） diff --git a/SKILL.md b/SKILL.md index 33c018c..d5c4cc2 100644 --- a/SKILL.md +++ b/SKILL.md @@ -1,6 +1,6 @@ --- name: video-edit-planner -description: "Use when planning video edits: transcribe audio, extract key frames, analyze visuals, and produce cut/transition/assembly plans via iterative dialogue." +description: "Use when planning video edits: transcribe audio, extract key frames, analyze visuals, and produce cut/transition/assembly plans via iterative dialogue. Defaults to subtractive editing (delete boring parts from continuous timeline) for coherent results." license: MIT tags: [ @@ -22,6 +22,18 @@ A conversational video-editing planning assistant. The user provides a video fil The skill is **agent-driven, not scripted**: it provides tools (transcription, clipping, frame extraction, index management) and guidance for the agent to autonomously decide when and what to extract based on user needs. +## Editing Strategy + +The skill defaults to **subtractive editing** — the most reliable way to produce coherent video from long recordings. + +**Subtractive (default):** Load the full recording onto the NLE timeline and delete boring parts. What remains is a continuous, naturally flowing video. The transcript is used to **identify what to cut** (sparse dialogue, repetitive actions, dead air), not what to extract. Best for: story-driven content, medium-length videos (5–15 min), gameplay with narrative arcs. + +**Additive (supplementary):** Extract individual interesting moments and assemble them from scratch. Best for: Shorts/TikTok, pure highlight reels, very short videos (<3 min). Produces short fragments that are hard to transition smoothly — use only when the target format demands it. + +**Hybrid (recommended for most cases):** Subtractive backbone with optional inserts. Delete boring parts from the continuous timeline, then optionally insert high-impact moments from elsewhere in the recording as cutaways. This combines coherence with the ability to spotlight great moments that occur outside the main narrative flow. + +When the user asks for a full edit plan, ask which strategy they prefer. Default to subtractive unless the user specifically wants Shorts/highlight-reel format. + ## When to Use - User provides a video path and wants help planning cuts, transitions, or assembly. @@ -70,6 +82,12 @@ Before transcribing, ask the user what they want to do with the video. User requ For full video-edit planning requests, ask early whether the user wants **one focused recommended cut** or **multiple alternative editing strategies**. If they want alternatives, analyze a wider set of candidate segments and present modular options the user can freely combine into their own final cut. Do not force the plan into a rigid one-path timeline where every section has only one fixed choice. +Also ask which editing strategy the user prefers: +- **Subtractive (default):** Delete boring parts from the full recording, keeping continuous segments. Best for coherence and flow. +- **Additive:** Extract individual clips and assemble them. Best for Shorts/highlight reels. +- **Hybrid (recommended):** Subtractive backbone with optional inserts from elsewhere. +If the user doesn't specify, default to subtractive — it produces more coherent results and is easier to execute in an NLE. + If the agent's runtime supports structured questioning (e.g., a `clarify` tool or a `grill-me` skill), use it to gather requirements systematically. Ask questions one at a time with multiple-choice options — this is much more effective than a free-text wall of questions. **The requirements questionnaire below covers the key axes for gameplay/creator video editing.** Not every question applies to every session — pick the relevant ones based on the user's initial request. See `references/editing-requirements-questionnaire.md` for the full questionnaire with choices. @@ -79,6 +97,7 @@ If the agent's runtime supports structured questioning (e.g., a `clarify` tool o | Target platform (Bilibili / Douyin / YouTube / Shorts) | Determines aspect ratio, pacing, subtitle density, meme density | | Narrative style (funny compilation / story + comedy / highlights montage) | Determines structure: pure clip reel vs. narrative arc | | Planning mode (single recommended cut / multiple alternatives) | Determines whether to produce one focused timeline or modular strategy options the user can mix and match | +| Editing strategy (subtractive / additive / hybrid) | Determines whether to plan by deleting boring parts from continuous timeline (default) or by extracting and assembling clips | | Role distribution (main POV / equal co-op / host / commentator) | Determines whose voice/reactions to prioritize in cuts | | Effect density (minimal / medium / high) | Determines how many stickers, overlays, and effects to search for and apply | | Editor tool (Premiere Pro / CapCut / DaVinci / other) | Determines how specific transition/effect instructions are phrased | @@ -333,6 +352,8 @@ Use whatever vision capability the agent's runtime provides to analyze extracted When the user asks for **clip candidates from a specific time range** of an already-transcribed video (for example, `22:00–45:00`) rather than a full edit plan for the whole recording, do a lightweight **transcript-first mining pass** before any heavy extraction: +**Subtractive framing:** When using a subtractive strategy, the transcript mining pass identifies BORING ranges to cut, not just interesting ranges to keep. A range with sparse dialogue, repetitive actions, or no reactions is a cut candidate. A range with dense dialogue, reactions, or story progression is a keep candidate. + 1. Load every relevant transcript track covering that window and slice to the requested range. 2. Compute simple density signals per minute or rolling window (segment count, character count, keyword hits, overlapping reactions across tracks). 3. Print and review the top windows plus manually chosen story pivots (setup / escalation / payoff), not just the numerically densest span. @@ -355,53 +376,53 @@ Output a Markdown table with overall recommendations, then let the user iterate. <1-3 sentences of high-level recommendation addressing the user's goal> -### Segments +### Editing strategy -Each segment below is self-contained: timecode, suggested duration, content/reason, edit technique, visual materials, sound effects, and BGM are all listed together — not split into separate chapters. + -#### Segment 1 — +### Timeline + +The timeline below shows what to KEEP and what to CUT. In subtractive mode, the "keep" segments form the backbone — the user loads the full recording into their NLE and deletes the "cut" ranges. In hybrid mode, "insert" segments are moments from elsewhere to drop into the timeline. + +#### Keep — | Field | Value | |---|---| -| Timecode | 00:01:23–00:01:45 | -| Suggested duration | ~22s | -| Content / reason | Character enters town, NPC dialogue reveals quest hook | -| Edit technique | Hard cut in, match-on-action to next segment | -| Visual materials | Sticker: shocked reaction face; search: shocked / surprised meme | -| Sound effects | Whoosh on cut; search: whoosh / transition sound effect | -| BGM | Light exploration theme, fade in at 00:01:25 | +| Timecode | 00:01:23–00:03:45 | +| Suggested duration | ~2m22s | +| Content / reason | Combat encounter with multi-phase boss, includes reactions and callouts | +| Edit technique | Keep continuous; trim start/end by ±3s for tighter entry | +| Visual materials | Zoom-in on phase transition; search: zoom-in / magnify meme | +| Sound effects | Impact hit on phase change; search: impact / hit sound effect | +| BGM | Combat track, sync phase change to beat drop | -#### Segment 2 — +#### Cut — | Field | Value | |---|---| -| Timecode | 00:01:45–00:02:10 | -| Suggested duration | ~25s | -| Content / reason | Walking with no dialogue or event — cut | -| Edit technique | Cut entirely | -| Visual materials | — | -| Sound effects | — | -| BGM | — | +| Timecode | 00:03:45–00:08:20 | +| Duration | ~4m35s | +| Reason | Walking between areas, no dialogue or events | +| Bridge shot | Optional: 0.5-1s fast-forward clip if the transition feels abrupt | -#### Segment 3 — +#### Insert (optional, hybrid mode) — | Field | Value | |---|---| -| Timecode | 00:02:10–00:02:35 | -| Suggested duration | ~25s | -| Content / reason | Combat starts, highlight moment with reaction | -| Edit technique | Keep, slow-motion on hit, fade in from black | -| Visual materials | Zoom-in effect on impact; search: zoom-in / magnify meme | -| Sound effects | Impact hit + record scratch on slow-mo; search: impact / punch / hit sound effect / record scratch | -| BGM | Switch to combat track, sync hit to beat drop | +| Source timecode | 00:45:10–00:45:25 | +| Suggested duration | ~15s | +| Content / reason | Funny reaction from later in the recording, works as a cutaway | +| Insert at | After the first keep segment (00:03:45) | +| Edit technique | Cutaway, audio dip under BGM | +| Visual materials | Reaction sticker; search: shocked / laughing meme | +| Sound effects | Record scratch on cutaway; search: record scratch | ### Alternative Strategy Options (if requested) | Module | Option | Best for | Candidate segments | Tradeoffs | | ------ | ------ | -------- | ------------------ | --------- | -| Opening | Cold open with funniest quote | Fast hook | 00:03:10–00:03:24 / 00:18:40–00:19:05 | Strong hook, weaker story setup | -| Story arc | Slow setup → escalation → payoff | Clear narrative | 00:01:20–00:02:30 / 00:12:00–00:14:10 / 00:22:30–00:23:20 | Better story, slower pacing | -| Insert | Short visual gag bridge | Rhythm reset | 00:08:10–00:08:16 | Optional; use only if pacing feels too dense | +| Opening | Cold open with funniest quote | Fast hook | 00:03:10–00:03:24 | Strong hook, weaker story setup | +| ... | ... | ... | ... | ... | ``` **Completion criteria:** user receives a Markdown table addressing their stated goal, and is invited to ask follow-up questions. @@ -555,6 +576,8 @@ This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user 26. **Saving frames to `/tmp/` or other temporary directories.** All extracted frames must go in the video file's directory (e.g., `_frames/`). When using `execute_code` or direct `ffmpeg` commands, always set the output path to the video directory — not `/tmp/frames/` or any other temp directory. Temporary directories are not portable, get cleaned up by the OS, and break the self-contained workflow. This applies to both spot-verification frames and full clip extraction. 27. **Using shell `&` to parallelize ffmpeg commands.** Some terminal tools reject shell-level backgrounding (`command &`) and require using `background=true` instead. When extracting multiple spot-verification frames in parallel, use a sequential `for` loop (each frame takes <1s so the total is still fast), or use the terminal tool's `background=true` parameter if available. Do not use `&` inside a single terminal call. 28. **Forgetting `--video` on `manage_index.py add-transcription`.** The `add-transcription` subcommand requires `--video VIDEO_PATH` to locate the index file next to the video. Omitting it causes an argument error. The same applies to `add-clip` and `add-frames` — always pass `--video`. +29. **Defaulting to additive editing for long-form content.** For videos longer than 3–4 minutes, subtractive editing (deleting boring parts from a continuous timeline) produces more coherent results than extracting and stitching short clips. The agent should default to subtractive unless the user explicitly wants Shorts/TikTok format or a pure highlight reel. See "Editing Strategy" section above. +30. **Recommending keep segments that are too short for subtractive editing.** In subtractive mode, "keep" segments should be continuous ranges of 30 seconds or longer — not 5–10 second fragments. If a 10-second moment is interesting but surrounded by boring content, recommend keeping a wider range (e.g., 30–60s) that includes setup and payoff, rather than extracting just the punchline. Short fragments are for "insert" segments in hybrid mode, not for the backbone timeline. ## Verification checklist diff --git a/references/editing-requirements-questionnaire.md b/references/editing-requirements-questionnaire.md index 15e14b1..1b969cd 100644 --- a/references/editing-requirements-questionnaire.md +++ b/references/editing-requirements-questionnaire.md @@ -21,28 +21,35 @@ Determines structure: pure clip reel vs. narrative arc. - Story + comedy hybrid: keep a simple story line and weave funny moments into it - Highlights montage: emphasize strong plays, clutch moments, wipes, reversals, or hype sections -### 3. Planning mode +### 3. Editing strategy +Determines whether to plan by deleting boring parts or by extracting and assembling clips. + +- Subtractive: delete boring parts from the full recording, keep continuous segments (default, best for coherence) +- Additive: extract individual clips and assemble them (best for Shorts/highlight reels) +- Hybrid: subtractive backbone with optional inserts from elsewhere (recommended for most cases) + +### 4. Planning mode Determines whether to produce one focused timeline or multiple modular alternatives. - Give me one strongest recommended full edit plan that is as directly executable as possible - Give me multiple alternative strategies so I can mix and match pieces into my own final cut - Give me one mainline plan plus optional replacement openings, endings, pacing choices, and insert ideas -### 4. Role distribution +### 5. Role distribution Determines whose voice/reactions to prioritize. - The uploader is the main point-of-view / main narrator; others mainly support the progression - Everyone is equally important; whoever has the funniest line or play gets the focus - The uploader is the host / commentator; their reactions and commentary drive the pacing -### 5. Effect density +### 6. Effect density Determines how many stickers, overlays, and effects to recommend. - Minimal: mostly natural cuts and transitions; no extra stickers or meme overlays unless essential - Medium: reinforce major punchlines with stickers/memes; visible transitions but not overly flashy - High: heavily stylized; frequent zooms, shakes, stickers, subtitles-as-effects, meme inserts, and visual exaggeration -### 6. Editor tool +### 7. Editor tool Determines how specific transition/effect instructions are phrased. - Premiere Pro @@ -50,14 +57,14 @@ Determines how specific transition/effect instructions are phrased. - DaVinci Resolve - Bcut -### 7. First edit or existing draft +### 8. First edit or existing draft Determines whether to plan from scratch or optimize existing cuts. - This is the first edit; plan the structure from scratch - I already have a rough cut / draft and want optimization suggestions - I already marked some candidate segments and want help connecting them -### 8. BGM style +### 9. BGM style Determines BGM search/recommendation direction. - Recommend automatically based on the content (light/funny for daily scenes, hype for combat/high-stakes sections) @@ -65,14 +72,14 @@ Determines BGM search/recommendation direction. - Mostly comedic / rhythmic / punchline-driven - Mostly hype / battle / high-energy -### 9. Number of participants +### 10. Number of participants Determines how to handle multi-person interaction and voice allocation. - 2-3 people (small co-op group) - 4-5 people (medium team) - 6-8 people (full large group) -### 10. Audio handling +### 11. Audio handling Determines subtitle work, BGM vs voice balance, and narration needs. - Keep original audio and add subtitles @@ -80,21 +87,21 @@ Determines subtitle work, BGM vs voice balance, and narration needs. - Replace much of the original audio with narration/voice-over; keep only the best live reactions - Mixed approach: mostly keep original audio with subtitles, but add narration for key sections -### 11. Privacy +### 12. Privacy Determines face-cam handling and any masking needs. - Face and voice can both be used normally - Do not show face, but voice is fine - Need masking / censoring / voice change processing -### 12. Output format +### 13. Output format Determines deliverable detail level. - Simple timestamp notes (for example, "00:12:30 this moment is very funny") - Detailed editing script table (timecode, segment description, why to keep it, transition ideas, asset needs) - Execution-ready storyboard / assembly plan (entry point, exit point, transition choice, asset placement, BGM timeline) -### 13. Target duration +### 14. Target duration Determines how aggressively to cut and pace the content. - User specifies a target duration (for example 7-8 minutes, 10 minutes, 3 minutes) @@ -102,7 +109,7 @@ Determines how aggressively to cut and pace the content. ## Usage notes - Ask one question at a time. The runtime's clarify tool with multiple-choice options works best. -- Not all 13 questions are needed every session. If the user's initial request already answers some axes, skip those. -- The questions can be asked in any order, but platform → narrative style → planning mode → effect density → editor tool is a natural priority flow. +- Not all 14 questions are needed every session. If the user's initial request already answers some axes, skip those. +- The questions can be asked in any order, but platform → narrative style → editing strategy → planning mode → effect density → editor tool is a natural priority flow. - If the user wants a full edit plan, ask the planning-mode question early so you know whether to produce one recommended timeline or multiple alternative strategies. - If a `grill-me` skill is available, it can be loaded to add follow-up depth questions beyond this template.