nite/video-edit-planner-skill
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: output paths, WSL/PowerShell transcription, grill-me hint

Browse Source 
Step 4 — Transcribe audio:
- Explicit: all output goes in video's own directory, never ~/whisperx_output/
- Transcription commands given to user to execute, not run by agent
- WSL agents provide PowerShell commands with Windows paths
- VIDEO_DIR and SKILL_DIR placeholders explained

Long-running operations:
- Rewritten: generate command for user, not agent executes
- WSL → PowerShell with Windows paths
- Agent verifies output after user reports completion

Self-contained setup:
- uv sync also given to user; WSL → PowerShell variant

Step 3 — Ask editing requirements:
- Optional grill-me skill loading hint for structured interviewing

Pitfalls #11, #12:
- #11: never use ~/whisperx_output/ or separate output dirs
- #12: don't run transcription in agent's own environment
This commit is contained in:
nite
2026-07-14 13:28:46 +08:00
parent c771bdcfaf
commit 8b353c99bc
1 changed files with 20 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SKILL.md
+20 -6
View File
@@ -73,7 +73,7 @@ Before transcribing, ask the user what they want to do with the video. Examples:
- "Plan transitions between scenes"
- "Find the best 30-second clip for a short"
User requirements are often vague in the first pass — that's expected. The plan is refined iteratively.
User requirements are often vague in the first pass — that's expected. The plan is refined iteratively. If the agent's runtime supports loading additional skills and a skill for structured interviewing (e.g., `grill-me`) is available, the agent may load it to sharpen the user's requirements through targeted questions.
**Completion criteria:** user has described at least a rough editing goal.
@@ -81,7 +81,15 @@ User requirements are often vague in the first pass — that's expected. The pla
Use the bundled `funasr-script` in `scripts/transcription/`. **Prefer Fun-ASR-Nano** as the default transcription model — it has higher accuracy and sensitivity on Chinese audio. Use SenseVoice only for fast previews.
**First run** requires `uv sync` (installs funasr + torch, may take several minutes). Subsequent runs reuse the cached venv. Notify the user before first-time setup.
**First run** requires `uv sync` (installs funasr + torch, may take several minutes). Subsequent runs reuse the cached venv.
**All output files (transcription JSON, clips, frames) go in the same directory as the video file.** Never use `~/whisperx_output/` or any other separate output directory — the `--output-dir` flag should point to the video's own directory. This keeps all artifacts co-located and portable.
**Transcription can be long-running.** The agent should generate the transcription command and **give it to the user to execute manually**, rather than running it in the agent's own environment. This avoids blocking the conversation and allows the user to run it on a more powerful machine if needed.
**If the agent detects it is running in WSL**, it should provide PowerShell commands for the user to run on the Windows host, not Linux commands. For example, convert the SKILL_DIR and VIDEO_PATH to their Windows equivalents (e.g., `C:\Users\...` instead of `/mnt/c/...`).
Command templates (adjust paths for the user's environment):
```bash
# Recommended default: Fun-ASR-Nano (high quality, Chinese-optimized)
@@ -94,6 +102,8 @@ uv run --directory SKILL_DIR/scripts/transcription funasr-fast VIDEO_PATH --trac
uv run --directory SKILL_DIR/scripts/transcription funasr-nano VIDEO_PATH --list-tracks
```
Where `VIDEO_DIR` is the directory containing the video file, and `SKILL_DIR` is the skill installation directory.
**Multi-track transcription:** when the user provides multiple track indexes (e.g., track 1 = mic, track 2 = Discord), run the transcription command **once per track**. Each run produces a separate JSON file named `<video_stem>_track<INDEX>_<model>.json`. Record each transcription in the index (Step 5) with its corresponding track index. When answering user questions, the agent should cross-reference all available transcripts to understand the full conversation context.
**Skip if cached:** check the index file (Step 5) for existing transcription entries. If a record exists for the same track index and the video file hasn't changed, reuse it.
@@ -325,13 +335,13 @@ Some material sites may block automated access (anti-bot, Cloudflare, CAPTCHA).
Before any operation expected to take >1 minute (first-time `uv sync`, transcription of long videos, large frame extraction):
1. **Notify the user** that a long operation is about to start and give a rough time estimate.
2. Run the operation in a way that does not block the conversation.
3. Do not poll progress at high frequency — wait for completion notification rather than checking every few seconds.
1. **Generate the command and give it to the user to execute manually**, rather than running it in the agent's own environment. This avoids blocking the conversation and lets the user run it on a more powerful machine (e.g., a Windows host while the agent runs in WSL).
2. If the agent runs in WSL, provide PowerShell commands with Windows paths (e.g., `C:\Users\...` instead of `/mnt/c/...`) for the user to run on the Windows host.
3. After the user reports completion, the agent should verify the output exists before proceeding.
## Self-contained setup
The skill bundles its own transcription project under `scripts/transcription/`. On first use:
The skill bundles its own transcription project under `scripts/transcription/`. On first use, the user needs to run:
```bash
cd SKILL_DIR/scripts/transcription && uv sync
@@ -339,6 +349,8 @@ cd SKILL_DIR/scripts/transcription && uv sync
This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user has previously installed these packages via uv in other projects, uv's cache will reuse them — first-time cost is only the link step.
If the agent runs in WSL, it should provide the equivalent PowerShell command with Windows paths for the user to run on the Windows host.
## Common pitfalls
1. **Installing dependencies without consent.** Always ask the user first. If they decline, stop — the skill cannot proceed without ffmpeg, uv, and python3.
@@ -351,6 +363,8 @@ This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user
8. **Hardcoding tool names.** This skill is agent-agnostic. Do not assume specific MCP tools or APIs exist. Use whatever the runtime provides for vision analysis, background execution, and user notification.
9. **Hardcoding output language.** The edit plan table and all user-facing text must follow the user's language, not a fixed language. The English table in Step 8 is a template — translate columns and content to match the user's language at runtime.
10. **Dropping audio tracks during clip extraction.** The default clip command maps only `0:v:0` (video-only) for fast frame extraction. If the user plans to import the clip into a video editor, use `--all-streams` (or `-map 0`) to preserve all audio tracks. Always ask the user which they need.
11. **Using `~/whisperx_output/` or other separate output directories.** All artifacts (transcription JSON, clips, frames, index file) must go in the same directory as the video file. The `--output-dir` flag should always point to the video's own directory. This keeps everything co-located and portable.
12. **Running transcription commands in the agent's own environment.** Transcription is long-running and resource-intensive. Generate the command and give it to the user to execute on their host machine. If the agent runs in WSL, provide PowerShell commands with Windows paths.
## Verification checklist