- **Subtractive-first editing** — defaults to delete-boring-parts strategy for coherent timelines; supports additive and hybrid modes for Shorts/highlight reels
- **Agent-agnostic** — no platform-specific tool names; works with any agent framework (Hermes, Claude Code, Codex, etc.)
## Quick Start
@@ -55,7 +56,7 @@ For Hermes Agent specifically, clone into `~/.hermes/skills/media/`. For other a
```
1. Check dependencies (ffmpeg, uv, python3)
2. Gather inputs (video path + audio track index)
3. Ask editing requirements
3. Ask editing requirements (including subtractive vs additive strategy)
4. Transcribe audio (skip if cached in index)
5. Extract clips & frames on demand (agent decides when transcript is insufficient)
6. Analyze frames with vision model (binary-search-style sampling)
description:"Use when planning video edits: transcribe audio, extract key frames, analyze visuals, and produce cut/transition/assembly plans via iterative dialogue."
description:"Use when planning video edits: transcribe audio, extract key frames, analyze visuals, and produce cut/transition/assembly plans via iterative dialogue. Defaults to subtractive editing (delete boring parts from continuous timeline) for coherent results."
license:MIT
tags:
[
@@ -22,6 +22,18 @@ A conversational video-editing planning assistant. The user provides a video fil
The skill is **agent-driven, not scripted**: it provides tools (transcription, clipping, frame extraction, index management) and guidance for the agent to autonomously decide when and what to extract based on user needs.
## Editing Strategy
The skill defaults to **subtractive editing** — the most reliable way to produce coherent video from long recordings.
**Subtractive (default):** Load the full recording onto the NLE timeline and delete boring parts. What remains is a continuous, naturally flowing video. The transcript is used to **identify what to cut** (sparse dialogue, repetitive actions, dead air), not what to extract. Best for: story-driven content, medium-length videos (5–15 min), gameplay with narrative arcs.
**Additive (supplementary):** Extract individual interesting moments and assemble them from scratch. Best for: Shorts/TikTok, pure highlight reels, very short videos (<3 min). Produces short fragments that are hard to transition smoothly — use only when the target format demands it.
**Hybrid (recommended for most cases):** Subtractive backbone with optional inserts. Delete boring parts from the continuous timeline, then optionally insert high-impact moments from elsewhere in the recording as cutaways. This combines coherence with the ability to spotlight great moments that occur outside the main narrative flow.
When the user asks for a full edit plan, ask which strategy they prefer. Default to subtractive unless the user specifically wants Shorts/highlight-reel format.
## When to Use
- User provides a video path and wants help planning cuts, transitions, or assembly.
@@ -70,6 +82,12 @@ Before transcribing, ask the user what they want to do with the video. User requ
For full video-edit planning requests, ask early whether the user wants **one focused recommended cut** or **multiple alternative editing strategies**. If they want alternatives, analyze a wider set of candidate segments and present modular options the user can freely combine into their own final cut. Do not force the plan into a rigid one-path timeline where every section has only one fixed choice.
Also ask which editing strategy the user prefers:
- **Subtractive (default):** Delete boring parts from the full recording, keeping continuous segments. Best for coherence and flow.
- **Additive:** Extract individual clips and assemble them. Best for Shorts/highlight reels.
- **Hybrid (recommended):** Subtractive backbone with optional inserts from elsewhere.
If the user doesn't specify, default to subtractive — it produces more coherent results and is easier to execute in an NLE.
If the agent's runtime supports structured questioning (e.g., a `clarify` tool or a `grill-me` skill), use it to gather requirements systematically. Ask questions one at a time with multiple-choice options — this is much more effective than a free-text wall of questions.
**The requirements questionnaire below covers the key axes for gameplay/creator video editing.** Not every question applies to every session — pick the relevant ones based on the user's initial request. See `references/editing-requirements-questionnaire.md` for the full questionnaire with choices.
@@ -79,6 +97,7 @@ If the agent's runtime supports structured questioning (e.g., a `clarify` tool o
| Narrative style (funny compilation / story + comedy / highlights montage) | Determines structure: pure clip reel vs. narrative arc |
| Planning mode (single recommended cut / multiple alternatives) | Determines whether to produce one focused timeline or modular strategy options the user can mix and match |
| Editing strategy (subtractive / additive / hybrid) | Determines whether to plan by deleting boring parts from continuous timeline (default) or by extracting and assembling clips |
| Role distribution (main POV / equal co-op / host / commentator) | Determines whose voice/reactions to prioritize in cuts |
| Effect density (minimal / medium / high) | Determines how many stickers, overlays, and effects to search for and apply |
| Editor tool (Premiere Pro / CapCut / DaVinci / other) | Determines how specific transition/effect instructions are phrased |
@@ -211,8 +230,8 @@ Use `scripts/index/manage_index.py` for all CRUD operations:
# Initialize index file (idempotent)
uv run --directory SKILL_DIR python scripts/index/manage_index.py init --video VIDEO_PATH
# Add transcription record
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-transcription --json-path PATH --track-index N --duration S
# Add transcription record (requires --video to locate the index file)
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-transcription --video VIDEO_PATH --json-path PATH --track-index N --duration S
# Query existing transcription (all tracks)
uv run --directory SKILL_DIR python scripts/index/manage_index.py get-transcription
uv run --directory SKILL_DIR python scripts/index/manage_index.py get-transcription --track 1
# Add a clip record
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-clip --start S --end E --path PATH --reason "user asked about this segment"
# Add a clip record (requires --video)
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-clip --video VIDEO_PATH --start S --end E --path PATH --reason "user asked about this segment"
# Add frame records (batch from a directory)
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-frames --clip-id N --frames-dir DIR --method scene
# Add frame records (batch from a directory, requires --video)
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-frames --video VIDEO_PATH --clip-id N --frames-dir DIR --method scene
# List all clips and frames
uv run --directory SKILL_DIR python scripts/index/manage_index.py list
**Record every extracted clip and frame in the index** before proceeding.
**Spot-verification frames** (single frames extracted at ASR timestamps to verify transcript claims) are lightweight probes, not durable artifacts. They do **not** need to be recorded in the index unless the agent determines they will be reused in later turns. If the verification is a one-pass check, skip the index to avoid clutter. If the agent expects to revisit the same timestamps, record them so subsequent turns can find them.
**Completion criteria:** clips and frames exist on disk and are recorded in the index.
### Step 7 — Vision analysis
@@ -331,6 +352,8 @@ Use whatever vision capability the agent's runtime provides to analyze extracted
When the user asks for **clip candidates from a specific time range** of an already-transcribed video (for example, `22:00–45:00`) rather than a full edit plan for the whole recording, do a lightweight **transcript-first mining pass** before any heavy extraction:
**Subtractive framing:** When using a subtractive strategy, the transcript mining pass identifies BORING ranges to cut, not just interesting ranges to keep. A range with sparse dialogue, repetitive actions, or no reactions is a cut candidate. A range with dense dialogue, reactions, or story progression is a keep candidate.
1. Load every relevant transcript track covering that window and slice to the requested range.
2. Compute simple density signals per minute or rolling window (segment count, character count, keyword hits, overlapping reactions across tracks).
3. Print and review the top windows plus manually chosen story pivots (setup / escalation / payoff), not just the numerically densest span.
@@ -353,53 +376,53 @@ Output a Markdown table with overall recommendations, then let the user iterate.
<1-3 sentences of high-level recommendation addressing the user's goal>
### Segments
### Editing strategy
Each segment below is self-contained: timecode, suggested duration, content/reason, edit technique, visual materials, sound effects, and BGM are all listed together — not split into separate chapters.
<Subtractive / Hybrid / Additive — one sentence explaining why>
#### Segment 1 — <short title>
### Timeline
The timeline below shows what to KEEP and what to CUT. In subtractive mode, the "keep" segments form the backbone — the user loads the full recording into their NLE and deletes the "cut" ranges. In hybrid mode, "insert" segments are moments from elsewhere to drop into the timeline.
| Insert | Short visual gag bridge | Rhythm reset | 00:08:10–00:08:16 | Optional; use only if pacing feels too dense |
| Opening | Cold open with funniest quote | Fast hook | 00:03:10–00:03:24 | Strong hook, weaker story setup |
| ... | ... | ... | ... | ... |
```
**Completion criteria:** user receives a Markdown table addressing their stated goal, and is invited to ask follow-up questions.
@@ -551,6 +574,10 @@ This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user
24. **Recommending optional clips that are too fragmented to use.** Optional clip candidates with timecodes of only 3–15 seconds are often too short to form a coherent, flowable segment when inserted into a video. Prefer self-contained segments of 20+ seconds. When the only interesting moment in a time range is a single 5-second quote, recommend that the user expand the selection range in Premiere (include ±30–60 seconds of surrounding context) to form a usable segment. Note this explicitly in the optional clip library. Short merchant/dialogue segments (30–60s) tend to be more self-contained and usable than short combat fragments.
25. **Recommending visual materials without corresponding audio effects.** Each scene's material recommendations should include both visual search terms (for stickers, GIFs, overlays) AND audio/sound-effect search terms (for whoosh, record scratch, alarm, loot ping, etc.). The user needs both to execute the edit in Premiere.
26. **Saving frames to `/tmp/` or other temporary directories.** All extracted frames must go in the video file's directory (e.g., `<video_stem>_frames/`). When using `execute_code` or direct `ffmpeg` commands, always set the output path to the video directory — not `/tmp/frames/` or any other temp directory. Temporary directories are not portable, get cleaned up by the OS, and break the self-contained workflow. This applies to both spot-verification frames and full clip extraction.
27. **Using shell `&` to parallelize ffmpeg commands.** Some terminal tools reject shell-level backgrounding (`command &`) and require using `background=true` instead. When extracting multiple spot-verification frames in parallel, use a sequential `for` loop (each frame takes <1s so the total is still fast), or use the terminal tool's `background=true` parameter if available. Do not use `&` inside a single terminal call.
28. **Forgetting `--video` on `manage_index.py add-transcription`.** The `add-transcription` subcommand requires `--video VIDEO_PATH` to locate the index file next to the video. Omitting it causes an argument error. The same applies to `add-clip` and `add-frames` — always pass `--video`.
29. **Defaulting to additive editing for long-form content.** For videos longer than 3–4 minutes, subtractive editing (deleting boring parts from a continuous timeline) produces more coherent results than extracting and stitching short clips. The agent should default to subtractive unless the user explicitly wants Shorts/TikTok format or a pure highlight reel. See "Editing Strategy" section above.
30. **Recommending keep segments that are too short for subtractive editing.** In subtractive mode, "keep" segments should be continuous ranges of 30 seconds or longer — not 5–10 second fragments. If a 10-second moment is interesting but surrounded by boring content, recommend keeping a wider range (e.g., 30–60s) that includes setup and payoff, rather than extracting just the punchline. Short fragments are for "insert" segments in hybrid mode, not for the backbone timeline.
- Execution-ready storyboard / assembly plan (entry point, exit point, transition choice, asset placement, BGM timeline)
### 13. Target duration
### 14. Target duration
Determines how aggressively to cut and pace the content.
- User specifies a target duration (for example 7-8 minutes, 10 minutes, 3 minutes)
@@ -102,7 +109,7 @@ Determines how aggressively to cut and pace the content.
## Usage notes
- Ask one question at a time. The runtime's clarify tool with multiple-choice options works best.
- Not all 13 questions are needed every session. If the user's initial request already answers some axes, skip those.
- The questions can be asked in any order, but platform → narrative style → planning mode → effect density → editor tool is a natural priority flow.
- Not all 14 questions are needed every session. If the user's initial request already answers some axes, skip those.
- The questions can be asked in any order, but platform → narrative style → editing strategy → planning mode → effect density → editor tool is a natural priority flow.
- If the user wants a full edit plan, ask the planning-mode question early so you know whether to produce one recommended timeline or multiple alternative strategies.
- If a `grill-me` skill is available, it can be loaded to add follow-up depth questions beyond this template.
@@ -20,6 +20,16 @@ When verifying ASR transcript claims against actual video visuals, use this ligh
**Important:** Save frames to the video file's directory, never to `/tmp/` or other temporary paths.
**Parallel extraction:** Use a sequential `for` loop to extract multiple frames in one terminal call — each frame takes <1s so the total is still fast. Do not use shell `&` backgrounding inside a single terminal call; some terminal tools reject it.
2. **Send frames to vision analysis with targeted questions.** For each frame, ask about the specific transcript claim:
- "The transcript says 'X' at this time. Can you see X happening?"
- "Is there a boss/enemy visible? Is combat happening?"
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
nite