docs: per-segment edit plan template, context-based ASR inference, remove copywriting pitfall

SKILL.md

@@ -152,6 +152,8 @@ Always verify with `ffprobe` before transcribing — the user may have a differe
 
 **ASR transcript reliability:** FunASR transcripts are useful for locating material, understanding the rough plot, and finding candidate time ranges, but they are not an authoritative text source for final deliverables. Misrecognized words are common when the original track has unclear pronunciation, overlap, noise, or game audio. Similar-sounding words may be substituted (for example, Chinese `帐篷` may be transcribed as `账本`). Before using transcript text as final copy, subtitles, quotes, captions, or narration, re-listen to the original audio around that time range and correct the text manually. When uncertain, mark the wording as uncertain rather than treating the ASR output as exact.
 
+**ASR JSON format:** FunASR output JSON files contain a `segments` array where each segment has `start` and `end` timestamps in **milliseconds** (not seconds), plus `text` and `speaker` fields. When filtering segments by time range, convert `mm:ss` to milliseconds first (e.g., `49:30` = `2970000` ms). The top-level `audio_duration_s` field is in seconds. Always verify units before using timestamps for frame extraction.
+
 **Skip if cached:** check the index file (Step 5) for existing transcription entries. If a record exists for the same track index and the video file hasn't changed, reuse it.
 
 **Completion criteria:** a transcription JSON exists for every requested track, and each is recorded in the index. Each JSON contains `segments` with timestamps and text.
@@ -248,6 +250,22 @@ uv run --directory SKILL_DIR python scripts/index/manage_index.py clean
 4. If a sub-range needs deeper understanding, extract more frames from that sub-range.
 5. Once the relevant range is identified, send all frames in that range (in batches if needed) to vision.
 
+**Spot-verification at ASR-mentioned timestamps:**
+
+When verifying specific transcript claims (e.g., "is there really a boss fight at 49:30?"), skip clip extraction and directly extract single frames at the exact ASR-mentioned timestamps:
+
+```bash
+# Extract one frame at a specific timestamp (seconds)
+ffmpeg -y -ss TIMESTAMP -i VIDEO_PATH -frames:v 1 -q:v 3 OUTPUT.jpg
+```
+
+This is much faster than clip+scene-detection for verifying a handful of specific points. Send each frame to vision analysis with a targeted question about what the transcript claims is happening at that moment (e.g., "The transcript says '他二阶段了' — can you see a boss in phase 2?").
+
+Key lessons from spot-verification:
+- **ASR timestamps can be off by several seconds** from the visual event they describe. If a frame at the ASR timestamp doesn't show the claimed content, try ±5–10 seconds before concluding the claim is wrong.
+- **ASR proper nouns are unreliable.** Boss names, item names, and location names may be homophone errors (e.g., "沙漠巨凤" Desert Giant Phoenix was transcribed as "沙漠巨风" Desert Giant Wind). Treat proper nouns in transcripts as phonetic hints, not authoritative labels.
+- **Visual confirmation can upgrade or downgrade segment priority.** A "待核验 S" segment with confirmed boss visuals becomes confirmed S; a segment where the transcript claims action but frames show only menus/UI should be downgraded.
+
 **Clip extraction** (keyframe-accurate, `-c copy`):
 
 For frame extraction purposes, only the video stream is needed:
@@ -303,6 +321,7 @@ Use whatever vision capability the agent's runtime provides to analyze extracted
 - **Batch size** — depends on the model's context window. See `references/frame-extraction-guide.md` for token cost estimates at different resolutions.
 - **Windows setup troubleshooting** — see `references/windows-funasr-uv-setup.md` for Windows-host PowerShell transcription commands, Python wheel issues such as `editdistance` source builds, and PyTorch CUDA-vs-CPU build checks.
 - **How many frames** per analysis pass — use the binary-search approach from Step 6.
+- **Spot-verification** — when verifying specific transcript claims, see `references/spot-verification-workflow.md` for the single-frame extraction protocol.
 
 **Completion criteria:** the agent has enough visual understanding to answer the user's question or produce the requested edit plan.
 
@@ -334,14 +353,45 @@ Output a Markdown table with overall recommendations, then let the user iterate.
 
 <1-3 sentences of high-level recommendation addressing the user's goal>
 
-### Detailed Plan
+### Segments
 
-| Timecode          | Duration | Segment Description             | Action        | Transition | Notes              |
-| ----------------- | -------- | ------------------------------- | ------------- | ---------- | ------------------ |
-| 00:01:23–00:01:45 | 22s      | Character enters town, dialogue | Keep          | Hard cut   | Can add subtitles  |
-| 00:01:45–00:02:10 | 25s      | Walking, no dialogue or event   | Cut           | —          | —                  |
-| 00:02:10–00:02:35 | 25s      | Combat starts, highlight moment | Keep, slow-mo | Fade in    | Sync to BGM rhythm |
-| ...               | ...      | ...                             | ...           | ...        | ...                |
+Each segment below is self-contained: timecode, suggested duration, content/reason, edit technique, visual materials, sound effects, and BGM are all listed together — not split into separate chapters.
+
+#### Segment 1 — <short title>
+
+| Field | Value |
+|---|---|
+| Timecode | 00:01:23–00:01:45 |
+| Suggested duration | ~22s |
+| Content / reason | Character enters town, NPC dialogue reveals quest hook |
+| Edit technique | Hard cut in, match-on-action to next segment |
+| Visual materials | Sticker: "震惊" reaction face; search: 震惊 / surprised / shocked meme |
+| Sound effects | Whoosh on cut; search: whoosh / 转场音效 |
+| BGM | Light exploration theme, fade in at 00:01:25 |
+
+#### Segment 2 — <short title>
+
+| Field | Value |
+|---|---|
+| Timecode | 00:01:45–00:02:10 |
+| Suggested duration | ~25s |
+| Content / reason | Walking with no dialogue or event — cut |
+| Edit technique | Cut entirely |
+| Visual materials | — |
+| Sound effects | — |
+| BGM | — |
+
+#### Segment 3 — <short title>
+
+| Field | Value |
+|---|---|
+| Timecode | 00:02:10–00:02:35 |
+| Suggested duration | ~25s |
+| Content / reason | Combat starts, highlight moment with reaction |
+| Edit technique | Keep, slow-motion on hit, fade in from black |
+| Visual materials | Zoom-in effect on impact; search: 放大镜 / zoom meme |
+| Sound effects | Impact hit + record scratch on slow-mo; search: impact / punch / 打击音效 / record scratch |
+| BGM | Switch to combat track, sync hit to beat drop |
 
 ### Alternative Strategy Options (if requested)
 
@@ -364,6 +414,24 @@ The user will likely ask follow-ups: "what about this section?", "add a transiti
 
 **Completion criteria:** user is satisfied with the plan or explicitly ends the session.
 
+## Resuming a previous editing session
+
+When the user says "continue editing" or resumes a prior session, the agent should recover state before doing new work:
+
+1. **Search for the previous session** using `session_search` with the video filename and relevant keywords (e.g., video name, "edit plan", "transcription").
+2. **Check for existing artifacts** next to the video file:
+   - Transcription JSON files (`<video_stem>_track*_*.json`)
+   - Edit plan files (`*edit_plan*.md` or similar)
+   - Index file (`<video_stem>.vedit.json`)
+3. **Read the existing plan** to understand where the user left off and what was pending.
+4. **Ask the user what to continue with** — do not assume the next step. The user may want to verify segments, add optional clips, revise the structure, or start a fresh pass.
+5. **Reuse existing transcriptions** — do not re-transcribe unless the user explicitly asks or the video file has changed.
+
+Common resume patterns:
+- "Continue editing [video]" → check session history, find plan file, ask what to work on next.
+- "Update the plan" → read existing plan, apply user's feedback, write updated version.
+- "I selected some clips from the optional library" → read plan + optional library, integrate user's selections into the main timeline.
+
 ## Finding stock materials (optional, on demand)
 
 When the user needs editing materials — stickers, GIFs, meme video clips, B-roll — the agent can help search or provide links for the user to search themselves. This section is **only triggered when the user asks for materials**; it is never a mandatory step.
@@ -388,6 +456,25 @@ If the user communicates in Chinese, **recommend 爱给网 and B站素材酷 fir
 - The agent can either **search directly** (using web tools or API calls) or **suggest expanded search terms** for the user to search manually.
 - If the user only wants links, provide the source URLs above and let the user search themselves.
 
+### Sound effects and audio materials
+
+Each scene's material recommendations should include **both** visual search terms (stickers, GIFs, overlays) **and** audio/sound-effect search terms. The user needs both to execute the edit in their NLE.
+
+Common sound-effect categories for gameplay/creator edits:
+
+| Category | Example search terms (expand by language) |
+|---|---|
+| Whoosh / transition | "whoosh", "swoosh", "风声", "转场音效" |
+| Record scratch / stop | "record scratch", "vinyl stop", "唱片刮擦" |
+| Impact / hit | "impact", "punch", "slap", "打击音效", "拳到肉" |
+| Alert / alarm | "alert", "alarm", "siren", "警报音效" |
+| Loot / pickup | "loot ping", "item pickup", "拾取音效", "开箱音效" |
+| Comedy / meme | "meme sound", "vine boom", "搞笑音效", "梗音效" |
+| Heartbeat / tension | "heartbeat", "tension", "心跳加速", "紧张音效" |
+| Level up / success | "level up", "fanfare", "升级音效", "成功音效" |
+
+Recommend sound effects per scene alongside visual materials, not in a separate chapter. See pitfall #24 about splitting advice into separate top-level chapters.
+
 ### B站素材酷 special handling
 
 B站素材酷 (cool.bilibili.com) has no built-in search function. Two options:
@@ -454,6 +541,16 @@ This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user
 14. **Saying you'll provide a command but not including it.** If you mention giving the user a transcription command, include it immediately in that same response. Do not move on to other content without printing the actual command. The user should never have to ask for it.
 15. **Sending a wall of free-text questions.** When gathering editing requirements, use structured one-at-a-time multiple-choice questions via the runtime's clarify/prompt tool. This is far more effective than dumping 10 questions as a paragraph.
 16. **Making full edit plans too rigid.** For complete planning requests, ask whether the user wants multiple alternative strategies. If yes, provide modular options and extra candidate segments that the user can combine, rather than a single fixed linear plan where every section has exactly one prescribed choice.
+17. **Assuming ASR timestamps are in seconds.** FunASR JSON segment `start`/`end` fields are in **milliseconds**. Filtering by `2970` (seconds) when the data is `2970000` (ms) silently returns zero results. Always check the first segment's timestamp magnitude before filtering.
+18. **Trusting ASR timestamps as frame-exact.** ASR segment timestamps can be off by several seconds from the visual event they describe. When doing spot-verification, if the frame at the ASR timestamp doesn't match the transcript claim, try ±5–10 seconds before concluding the claim is wrong.
+19. **Treating ASR proper nouns as authoritative.** Boss names, item names, location names, and character names in ASR transcripts may be homophone errors (e.g., "沙漠巨凤" → "沙漠巨风", "帐篷" → "账本"). Use transcript proper nouns as phonetic hints for searching, but verify against visual/UI evidence before using them in final copy.
+20. **Assuming transcript-claimed action is visually present.** A transcript saying "鸟追着打" (bird chasing) does not guarantee the bird is visible on screen at that timestamp — the player may be in a menu, or the action may have happened off-screen. Always spot-verify with frames before promising visual content in the edit plan. Distinguish "transcript-grounded" from "visually confirmed" segments in the plan.
+21. **Inferring psychological state from isolated ASR sentences.** ASR transcripts capture *what was said*, not *how it was said* — tone, emotion, and situational context are invisible in the text. A phrase like "笑死" (laugh to death) might mean the speaker is speechless/frustrated (无语), not genuinely amused. "你要害死我" (you're going to kill me) might be about one person entering a dungeon room first while the other stopped to eat a healing item — a coordination timing issue, not blame. Inferring psychological state or intent from context IS allowed, but it must be based on surrounding dialogue and situational context, not a single sentence in isolation. When a single transcript line could be misread, check nearby segments for context, ask the user for clarification, or mark the description as uncertain.
+22. **Including non-game content in a game edit plan.** If the video is a gameplay recording, the edit plan should focus on game-related content only. Off-topic segments (microphone crosstalk, refund discussions, personal banter unrelated to the game) should not be included unless the user specifically requests them. The user's chat language being Chinese does not mean they want non-game Chinese-language banter in the plan.
+23. **Splitting technique/BGM/material advice into separate top-level chapters.** The edit-plan document should be organized around the script table (each scene/segment as a subsection), with BGM search terms, sound-effect search terms, visual-material search terms, Premiere technique notes (放大镜, 屏幕震动, 快速回放), and any other per-scene guidance embedded directly under each subsection — NOT split into separate top-level sections (e.g., "§4 Premiere 技巧", "§5 素材关键词", "§6 BGM 策略"). The user reads the plan scene-by-scene; scattering technique advice across detached chapters forces them to cross-reference.
+24. **Recommending optional clips that are too fragmented to use.** Optional clip candidates with timecodes of only 3–15 seconds are often too short to form a coherent, flowable segment when inserted into a video. Prefer self-contained segments of 20+ seconds. When the only interesting moment in a time range is a single 5-second quote, recommend that the user expand the selection range in Premiere (include ±30–60 seconds of surrounding context) to form a usable segment. Note this explicitly in the optional clip library. Short merchant/dialogue segments (30–60s) tend to be more self-contained and usable than short combat fragments.
+25. **Recommending visual materials without corresponding audio effects.** Each scene's material recommendations should include both visual search terms (for stickers, GIFs, overlays) AND audio/sound-effect search terms (for whoosh, record scratch, alarm, loot ping, etc.). The user needs both to execute the edit in Premiere.
+26. **Saving frames to `/tmp/` or other temporary directories.** All extracted frames must go in the video file's directory (e.g., `<video_stem>_frames/`). When using `execute_code` or direct `ffmpeg` commands, always set the output path to the video directory — not `/tmp/frames/` or any other temp directory. Temporary directories are not portable, get cleaned up by the OS, and break the self-contained workflow. This applies to both spot-verification frames and full clip extraction.
 
 ## Verification checklist
 

references/spot-verification-workflow.md

@@ -0,0 +1,56 @@
+# Spot-Verification Workflow
+
+When verifying ASR transcript claims against actual video visuals, use this lightweight protocol instead of full clip extraction.
+
+## When to use
+
+- You have a transcript with timestamps and need to confirm what's visually happening at specific moments.
+- You need to upgrade/downgrade segment priority based on visual evidence.
+- You want to check whether a transcript-claimed event (boss fight, chest opening, character death) actually has corresponding on-screen visuals.
+
+## Procedure
+
+1. **Extract frames at ASR-mentioned timestamps.** For each key transcript claim, extract a single frame:
+
+   ```bash
+   ffmpeg -y -ss <TIMESTAMP_IN_SECONDS> -i <VIDEO_PATH> -frames:v 1 -q:v 3 /tmp/frames/frame_<MM_SS>.jpg
+   ```
+
+   Use `-ss` before `-i` for fast keyframe seeking. This takes <1s per frame.
+
+2. **Send frames to vision analysis with targeted questions.** For each frame, ask about the specific transcript claim:
+   - "The transcript says 'X' at this time. Can you see X happening?"
+   - "Is there a boss/enemy visible? Is combat happening?"
+   - "Is there a [specific item/UI] visible?"
+
+3. **Handle timestamp discrepancies.** If the frame doesn't match the transcript claim:
+   - Try ±5–10 seconds around the timestamp.
+   - ASR timestamps can lag or lead the visual event by several seconds.
+   - If still no match after ±10s, the claim may be about off-screen action or voice-only commentary.
+
+4. **Record verification results.** For each verified segment, note:
+   - Confirmed: visual matches transcript claim → keep or upgrade priority
+   - Discrepancy: visual differs from transcript (e.g., menu open instead of combat) → downgrade or reclassify
+   - Not found: no visual evidence for the claimed content → mark as "voice-only" or "needs Premiere manual check"
+
+5. **Distinguish in the edit plan.** Label each segment as:
+   - **Visually confirmed** — frames show the claimed content
+   - **Transcript-grounded** — only ASR evidence; visual not yet checked
+   - **Needs manual check** — frames inconclusive; must be verified in Premiere
+
+## Concrete example (Romestead session 2026-07-14)
+
+| ASR timestamp | Transcript claim | Frame result | Verification |
+|---|---|---|---|
+| 49:08 | "他跟第一关的boss怎么一样" (boss comparison) | Boss "沙漠巨凤" visible with health bar, combat active | ✅ Confirmed S |
+| 50:56 | "他二阶段了" (phase 2) | Boss visible, health bar nearly depleted, intense effects | ✅ Confirmed |
+| 51:26 | "他打我有点疼" (hits hard) | Boss visible, player HP low (~20-30%), damage numbers | ✅ Confirmed |
+| 52:28 | "打死了" (killed) | Boss still visible at 52:28; boss gone by 52:40 | ⚠️ Kill ~52:30-52:40 |
+| 55:35 | "鸟追着打" (bird chasing) | Player in base/village with inventory open, no bird visible | ❌ Not confirmed |
+
+## Key lessons
+
+- ASR segment `start`/`end` are in **milliseconds**, not seconds. Convert before filtering.
+- ASR proper nouns are phonetic hints: "沙漠巨凤" (Phoenix) was transcribed as "沙漠巨风" (Wind). Verify names against on-screen UI.
+- "打死了" (killed it) at 52:28 was actually ~52:35–52:40 visually — a 7–12 second offset.
+- "鸟追着打" (bird chase) had zero visual evidence in the 55:00–56:00 range — the player was in a menu. The "凤凰之翼" (Phoenix Wing) item existed in inventory, but the chase itself was voice-only commentary or happened earlier.