fix: add --video to manage_index examples, spot-verification parallel loop
This commit is contained in:
@@ -211,8 +211,8 @@ Use `scripts/index/manage_index.py` for all CRUD operations:
|
||||
# Initialize index file (idempotent)
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py init --video VIDEO_PATH
|
||||
|
||||
# Add transcription record
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-transcription --json-path PATH --track-index N --duration S
|
||||
# Add transcription record (requires --video to locate the index file)
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-transcription --video VIDEO_PATH --json-path PATH --track-index N --duration S
|
||||
|
||||
# Query existing transcription (all tracks)
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py get-transcription
|
||||
@@ -220,11 +220,11 @@ uv run --directory SKILL_DIR python scripts/index/manage_index.py get-transcript
|
||||
# Query specific track
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py get-transcription --track 1
|
||||
|
||||
# Add a clip record
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-clip --start S --end E --path PATH --reason "user asked about this segment"
|
||||
# Add a clip record (requires --video)
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-clip --video VIDEO_PATH --start S --end E --path PATH --reason "user asked about this segment"
|
||||
|
||||
# Add frame records (batch from a directory)
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-frames --clip-id N --frames-dir DIR --method scene
|
||||
# Add frame records (batch from a directory, requires --video)
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py add-frames --video VIDEO_PATH --clip-id N --frames-dir DIR --method scene
|
||||
|
||||
# List all clips and frames
|
||||
uv run --directory SKILL_DIR python scripts/index/manage_index.py list
|
||||
@@ -311,6 +311,8 @@ ffmpeg -hide_banner -y -i CLIP_PATH \
|
||||
|
||||
**Record every extracted clip and frame in the index** before proceeding.
|
||||
|
||||
**Spot-verification frames** (single frames extracted at ASR timestamps to verify transcript claims) are lightweight probes, not durable artifacts. They do **not** need to be recorded in the index unless the agent determines they will be reused in later turns. If the verification is a one-pass check, skip the index to avoid clutter. If the agent expects to revisit the same timestamps, record them so subsequent turns can find them.
|
||||
|
||||
**Completion criteria:** clips and frames exist on disk and are recorded in the index.
|
||||
|
||||
### Step 7 — Vision analysis
|
||||
@@ -551,6 +553,8 @@ This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user
|
||||
24. **Recommending optional clips that are too fragmented to use.** Optional clip candidates with timecodes of only 3–15 seconds are often too short to form a coherent, flowable segment when inserted into a video. Prefer self-contained segments of 20+ seconds. When the only interesting moment in a time range is a single 5-second quote, recommend that the user expand the selection range in Premiere (include ±30–60 seconds of surrounding context) to form a usable segment. Note this explicitly in the optional clip library. Short merchant/dialogue segments (30–60s) tend to be more self-contained and usable than short combat fragments.
|
||||
25. **Recommending visual materials without corresponding audio effects.** Each scene's material recommendations should include both visual search terms (for stickers, GIFs, overlays) AND audio/sound-effect search terms (for whoosh, record scratch, alarm, loot ping, etc.). The user needs both to execute the edit in Premiere.
|
||||
26. **Saving frames to `/tmp/` or other temporary directories.** All extracted frames must go in the video file's directory (e.g., `<video_stem>_frames/`). When using `execute_code` or direct `ffmpeg` commands, always set the output path to the video directory — not `/tmp/frames/` or any other temp directory. Temporary directories are not portable, get cleaned up by the OS, and break the self-contained workflow. This applies to both spot-verification frames and full clip extraction.
|
||||
27. **Using shell `&` to parallelize ffmpeg commands.** Some terminal tools reject shell-level backgrounding (`command &`) and require using `background=true` instead. When extracting multiple spot-verification frames in parallel, use a sequential `for` loop (each frame takes <1s so the total is still fast), or use the terminal tool's `background=true` parameter if available. Do not use `&` inside a single terminal call.
|
||||
28. **Forgetting `--video` on `manage_index.py add-transcription`.** The `add-transcription` subcommand requires `--video VIDEO_PATH` to locate the index file next to the video. Omitting it causes an argument error. The same applies to `add-clip` and `add-frames` — always pass `--video`.
|
||||
|
||||
## Verification checklist
|
||||
|
||||
|
||||
@@ -20,6 +20,16 @@ When verifying ASR transcript claims against actual video visuals, use this ligh
|
||||
|
||||
**Important:** Save frames to the video file's directory, never to `/tmp/` or other temporary paths.
|
||||
|
||||
**Parallel extraction:** Use a sequential `for` loop to extract multiple frames in one terminal call — each frame takes <1s so the total is still fast. Do not use shell `&` backgrounding inside a single terminal call; some terminal tools reject it.
|
||||
|
||||
```bash
|
||||
for ts in "00:05:17" "00:14:06" "00:28:00"; do
|
||||
fn="$DIR/spot_${ts//:/}.jpg"
|
||||
ffmpeg -hide_banner -y -ss "$ts" -i "$VIDEO" -frames:v 1 -q:v 3 "$fn" 2>/dev/null
|
||||
echo "$ts -> $fn ($(stat -c%s "$fn" 2>/dev/null) bytes)"
|
||||
done
|
||||
```
|
||||
|
||||
2. **Send frames to vision analysis with targeted questions.** For each frame, ask about the specific transcript claim:
|
||||
- "The transcript says 'X' at this time. Can you see X happening?"
|
||||
- "Is there a boss/enemy visible? Is combat happening?"
|
||||
|
||||
Reference in New Issue
Block a user