nite/video-edit-planner-skill
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: warn ASR transcripts are not final copy

Browse Source
This commit is contained in:
nite
2026-07-14 16:38:26 +08:00
parent 98c478a1bb
commit 666ac0c288
1 changed files with 28 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SKILL.md
+28 -8
View File
@@ -144,6 +144,8 @@ Always verify with `ffprobe` before transcribing — the user may have a differe
**Note on Discord audio quality:** Discord tracks may contain noise, crosstalk, or non-participant chatter. This is normal — the ASR transcript will be noisier. When cross-referencing transcripts, treat Discord track content as supplementary context, not as the primary narrative source.
**ASR transcript reliability:** FunASR transcripts are useful for locating material, understanding the rough plot, and finding candidate time ranges, but they are not an authoritative text source for final deliverables. Misrecognized words are common when the original track has unclear pronunciation, overlap, noise, or game audio. Similar-sounding words may be substituted (for example, Chinese `帐篷` may be transcribed as `账本`). Before using transcript text as final copy, subtitles, quotes, captions, or narration, re-listen to the original audio around that time range and correct the text manually. When uncertain, mark the wording as uncertain rather than treating the ASR output as exact.
**Skip if cached:** check the index file (Step 5) for existing transcription entries. If a record exists for the same track index and the video file hasn't changed, reuse it.
**Completion criteria:** a transcription JSON exists for every requested track, and each is recorded in the index. Each JSON contains `segments` with timestamps and text.
@@ -300,6 +302,23 @@ Use whatever vision capability the agent's runtime provides to analyze extracted
### Step 8 — Produce edit plan
#### Transcript-first candidate mining for a bounded review window
When the user asks for **clip candidates from a specific time range** of an already-transcribed video (for example, `22:00–45:00`) rather than a full edit plan for the whole recording, do a lightweight **transcript-first mining pass** before any heavy extraction:
1. Load every relevant transcript track covering that window and slice to the requested range.
2. Compute simple density signals per minute or rolling window (segment count, character count, keyword hits, overlapping reactions across tracks).
3. Print and review the top windows plus manually chosen story pivots (setup / escalation / payoff), not just the numerically densest span.
4. Cross-read the main mic track with secondary chat/Discord tracks — use side tracks as corroboration and reaction context, not as the sole source of truth.
5. For the shortlisted windows, do a **spot visual check** with a few representative frames (for example 3–4 timestamps per candidate, optionally stitched into a contact sheet) instead of extracting long clips immediately.
6. Only extract full clips if the transcript and spot frames are still insufficient.
For multi-track recordings with crosstalk or duplicated speech, treat ASR as a **locator index**. Merge tracks by time, de-duplicate repeated lines mentally, and prefer moments where both dialogue and visuals support a self-contained mini-story. Secondary tracks may contain unrelated chatter; they are useful for reactions but should not dominate the narrative. When presenting the plan, call out which suggestions are transcript-grounded and which were visually spot-checked.
For fast-paced story + comedy gameplay videos, cull hard: keep combat, danger, reactions, relationship banter, and short setup/payoff beats; compress crafting, shopping, and inventory UI into 0.5–2 second bridge shots unless the item/choice itself is the punchline. A 7–8 minute plan should usually be a chaptered structure, not a flat list of every candidate.
This workflow is especially effective for requests like **“find fast-paced funny + story-driven candidate segments”**. It keeps token/runtime cost low while still verifying whether a transcript-highlighted moment actually has usable on-screen action.
Output a Markdown table with overall recommendations, then let the user iterate. **Respond in the same language the user is using.**
```markdown
@@ -411,14 +430,15 @@ This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user
4. **Using full-resolution frames.** A 2560×1600 PNG can cost 2000+ vision tokens. Downsample to 1280 width with `scale=1280:-2` and use JPEG (`-q:v 3`).
5. **Forgetting to record artifacts in the index.** Every clip and frame batch must be recorded. Otherwise subsequent turns will re-extract the same content.
6. **Assuming the transcript is sufficient.** Gameplay videos often have long stretches of action with no dialogue. The agent should proactively check whether visual analysis is needed for segments the user asks about.
7. **Mixing funasr-script flags.** The bundled `funasr-nano`/`funasr-fast` use `--track STREAM_INDEX`, explicit `--language AUDIO_LANGUAGE_CODE`, and `--output-dir DIR`. Do not use the legacy `~/.local/bin/funasr-transcribe` wrapper's flags.
8. **Hardcoding tool names.** This skill is agent-agnostic. Do not assume specific MCP tools or APIs exist. Use whatever the runtime provides for vision analysis, background execution, and user notification.
9. **Hardcoding output language.** The edit plan table and all user-facing text must follow the user's language, not a fixed language. The English table in Step 8 is a template — translate columns and content to match the user's language at runtime.
10. **Dropping audio tracks during clip extraction.** The default clip command maps only `0:v:0` (video-only) for fast frame extraction. If the user plans to import the clip into a video editor, use `--all-streams` (or `-map 0`) to preserve all audio tracks. Always ask the user which they need.
11. **Using `~/whisperx_output/` or other separate output directories.** All artifacts (transcription JSON, clips, frames, index file) must go in the same directory as the video file. The `--output-dir` flag should always point to the video's own directory. This keeps everything co-located and portable.
12. **Choosing command syntax or execution environment solely from where the agent runs.** Choose based on where the video file lives and where the command will execute. Windows execution → PowerShell commands and Windows paths; WSL/Linux execution → Linux shell commands and Linux paths. If the agent runs in WSL but Windows execution is chosen, use `powershell.exe` to do lightweight setup (copy files, install/check dependencies, run `uv sync`) before handing only the long transcription command to the user. Never ask Windows `uv` to run from `\\\\wsl.localhost\\...` / `\\\\wsl$\\...`.
13. **Saying you'll provide a command but not including it.** If you mention giving the user a transcription command, include it immediately in that same response. Do not move on to other content without printing the actual command. The user should never have to ask for it.
14. **Sending a wall of free-text questions.** When gathering editing requirements, use structured one-at-a-time multiple-choice questions via the runtime's clarify/prompt tool. This is far more effective than dumping 10 questions as a paragraph.
7. **Treating ASR text as final copy.** FunASR transcripts can contain wrong characters/words caused by unclear speech, noise, overlap, or similar-sounding terms (e.g., `帐篷` → `账本`). Use transcripts for locating material and understanding context, not as the absolute wording for final captions, quotes, narration, or published copy. Re-listen and manually correct any text that will appear in the final product.
8. **Mixing funasr-script flags.** The bundled `funasr-nano`/`funasr-fast` use `--track STREAM_INDEX`, explicit `--language AUDIO_LANGUAGE_CODE`, and `--output-dir DIR`. Do not use the legacy `~/.local/bin/funasr-transcribe` wrapper's flags.
9. **Hardcoding tool names.** This skill is agent-agnostic. Do not assume specific MCP tools or APIs exist. Use whatever the runtime provides for vision analysis, background execution, and user notification.
10. **Hardcoding output language.** The edit plan table and all user-facing text must follow the user's language, not a fixed language. The English table in Step 8 is a template — translate columns and content to match the user's language at runtime.
11. **Dropping audio tracks during clip extraction.** The default clip command maps only `0:v:0` (video-only) for fast frame extraction. If the user plans to import the clip into a video editor, use `--all-streams` (or `-map 0`) to preserve all audio tracks. Always ask the user which they need.
12. **Using `~/whisperx_output/` or other separate output directories.** All artifacts (transcription JSON, clips, frames, index file) must go in the same directory as the video file. The `--output-dir` flag should always point to the video's own directory. This keeps everything co-located and portable.
13. **Choosing command syntax or execution environment solely from where the agent runs.** Choose based on where the video file lives and where the command will execute. Windows execution → PowerShell commands and Windows paths; WSL/Linux execution → Linux shell commands and Linux paths. If the agent runs in WSL but Windows execution is chosen, use `powershell.exe` to do lightweight setup (copy files, install/check dependencies, run `uv sync`) before handing only the long transcription command to the user. Never ask Windows `uv` to run from `\\\\wsl.localhost\\...` / `\\\\wsl$\\...`.
14. **Saying you'll provide a command but not including it.** If you mention giving the user a transcription command, include it immediately in that same response. Do not move on to other content without printing the actual command. The user should never have to ask for it.
15. **Sending a wall of free-text questions.** When gathering editing requirements, use structured one-at-a-time multiple-choice questions via the runtime's clarify/prompt tool. This is far more effective than dumping 10 questions as a paragraph.
## Verification checklist