From 87eb3392df2b72f64c3e85c284bb606878685404 Mon Sep 17 00:00:00 2001 From: nite Date: Wed, 15 Jul 2026 15:23:23 +0800 Subject: [PATCH] fix: add --video to manage_index examples, spot-verification parallel loop --- SKILL.md | 16 ++++++++++------ references/spot-verification-workflow.md | 10 ++++++++++ 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/SKILL.md b/SKILL.md index b056f44..33c018c 100644 --- a/SKILL.md +++ b/SKILL.md @@ -211,8 +211,8 @@ Use `scripts/index/manage_index.py` for all CRUD operations: # Initialize index file (idempotent) uv run --directory SKILL_DIR python scripts/index/manage_index.py init --video VIDEO_PATH -# Add transcription record -uv run --directory SKILL_DIR python scripts/index/manage_index.py add-transcription --json-path PATH --track-index N --duration S +# Add transcription record (requires --video to locate the index file) +uv run --directory SKILL_DIR python scripts/index/manage_index.py add-transcription --video VIDEO_PATH --json-path PATH --track-index N --duration S # Query existing transcription (all tracks) uv run --directory SKILL_DIR python scripts/index/manage_index.py get-transcription @@ -220,11 +220,11 @@ uv run --directory SKILL_DIR python scripts/index/manage_index.py get-transcript # Query specific track uv run --directory SKILL_DIR python scripts/index/manage_index.py get-transcription --track 1 -# Add a clip record -uv run --directory SKILL_DIR python scripts/index/manage_index.py add-clip --start S --end E --path PATH --reason "user asked about this segment" +# Add a clip record (requires --video) +uv run --directory SKILL_DIR python scripts/index/manage_index.py add-clip --video VIDEO_PATH --start S --end E --path PATH --reason "user asked about this segment" -# Add frame records (batch from a directory) -uv run --directory SKILL_DIR python scripts/index/manage_index.py add-frames --clip-id N --frames-dir DIR --method scene +# Add frame records (batch from a directory, requires --video) +uv run --directory SKILL_DIR python scripts/index/manage_index.py add-frames --video VIDEO_PATH --clip-id N --frames-dir DIR --method scene # List all clips and frames uv run --directory SKILL_DIR python scripts/index/manage_index.py list @@ -311,6 +311,8 @@ ffmpeg -hide_banner -y -i CLIP_PATH \ **Record every extracted clip and frame in the index** before proceeding. +**Spot-verification frames** (single frames extracted at ASR timestamps to verify transcript claims) are lightweight probes, not durable artifacts. They do **not** need to be recorded in the index unless the agent determines they will be reused in later turns. If the verification is a one-pass check, skip the index to avoid clutter. If the agent expects to revisit the same timestamps, record them so subsequent turns can find them. + **Completion criteria:** clips and frames exist on disk and are recorded in the index. ### Step 7 — Vision analysis @@ -551,6 +553,8 @@ This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user 24. **Recommending optional clips that are too fragmented to use.** Optional clip candidates with timecodes of only 3–15 seconds are often too short to form a coherent, flowable segment when inserted into a video. Prefer self-contained segments of 20+ seconds. When the only interesting moment in a time range is a single 5-second quote, recommend that the user expand the selection range in Premiere (include ±30–60 seconds of surrounding context) to form a usable segment. Note this explicitly in the optional clip library. Short merchant/dialogue segments (30–60s) tend to be more self-contained and usable than short combat fragments. 25. **Recommending visual materials without corresponding audio effects.** Each scene's material recommendations should include both visual search terms (for stickers, GIFs, overlays) AND audio/sound-effect search terms (for whoosh, record scratch, alarm, loot ping, etc.). The user needs both to execute the edit in Premiere. 26. **Saving frames to `/tmp/` or other temporary directories.** All extracted frames must go in the video file's directory (e.g., `_frames/`). When using `execute_code` or direct `ffmpeg` commands, always set the output path to the video directory — not `/tmp/frames/` or any other temp directory. Temporary directories are not portable, get cleaned up by the OS, and break the self-contained workflow. This applies to both spot-verification frames and full clip extraction. +27. **Using shell `&` to parallelize ffmpeg commands.** Some terminal tools reject shell-level backgrounding (`command &`) and require using `background=true` instead. When extracting multiple spot-verification frames in parallel, use a sequential `for` loop (each frame takes <1s so the total is still fast), or use the terminal tool's `background=true` parameter if available. Do not use `&` inside a single terminal call. +28. **Forgetting `--video` on `manage_index.py add-transcription`.** The `add-transcription` subcommand requires `--video VIDEO_PATH` to locate the index file next to the video. Omitting it causes an argument error. The same applies to `add-clip` and `add-frames` — always pass `--video`. ## Verification checklist diff --git a/references/spot-verification-workflow.md b/references/spot-verification-workflow.md index 0a6b122..40cec30 100644 --- a/references/spot-verification-workflow.md +++ b/references/spot-verification-workflow.md @@ -20,6 +20,16 @@ When verifying ASR transcript claims against actual video visuals, use this ligh **Important:** Save frames to the video file's directory, never to `/tmp/` or other temporary paths. + **Parallel extraction:** Use a sequential `for` loop to extract multiple frames in one terminal call — each frame takes <1s so the total is still fast. Do not use shell `&` backgrounding inside a single terminal call; some terminal tools reject it. + + ```bash + for ts in "00:05:17" "00:14:06" "00:28:00"; do + fn="$DIR/spot_${ts//:/}.jpg" + ffmpeg -hide_banner -y -ss "$ts" -i "$VIDEO" -frames:v 1 -q:v 3 "$fn" 2>/dev/null + echo "$ts -> $fn ($(stat -c%s "$fn" 2>/dev/null) bytes)" + done + ``` + 2. **Send frames to vision analysis with targeted questions.** For each frame, ask about the specific transcript claim: - "The transcript says 'X' at this time. Can you see X happening?" - "Is there a boss/enemy visible? Is combat happening?"