From 8b353c99bccd7dd741bf5f104a93be2505ba48f0 Mon Sep 17 00:00:00 2001 From: nite Date: Tue, 14 Jul 2026 13:28:46 +0800 Subject: [PATCH] fix: output paths, WSL/PowerShell transcription, grill-me hint MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Step 4 — Transcribe audio: - Explicit: all output goes in video's own directory, never ~/whisperx_output/ - Transcription commands given to user to execute, not run by agent - WSL agents provide PowerShell commands with Windows paths - VIDEO_DIR and SKILL_DIR placeholders explained Long-running operations: - Rewritten: generate command for user, not agent executes - WSL → PowerShell with Windows paths - Agent verifies output after user reports completion Self-contained setup: - uv sync also given to user; WSL → PowerShell variant Step 3 — Ask editing requirements: - Optional grill-me skill loading hint for structured interviewing Pitfalls #11, #12: - #11: never use ~/whisperx_output/ or separate output dirs - #12: don't run transcription in agent's own environment --- SKILL.md | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/SKILL.md b/SKILL.md index 11317d1..9f256cf 100644 --- a/SKILL.md +++ b/SKILL.md @@ -73,7 +73,7 @@ Before transcribing, ask the user what they want to do with the video. Examples: - "Plan transitions between scenes" - "Find the best 30-second clip for a short" -User requirements are often vague in the first pass — that's expected. The plan is refined iteratively. +User requirements are often vague in the first pass — that's expected. The plan is refined iteratively. If the agent's runtime supports loading additional skills and a skill for structured interviewing (e.g., `grill-me`) is available, the agent may load it to sharpen the user's requirements through targeted questions. **Completion criteria:** user has described at least a rough editing goal. @@ -81,7 +81,15 @@ User requirements are often vague in the first pass — that's expected. The pla Use the bundled `funasr-script` in `scripts/transcription/`. **Prefer Fun-ASR-Nano** as the default transcription model — it has higher accuracy and sensitivity on Chinese audio. Use SenseVoice only for fast previews. -**First run** requires `uv sync` (installs funasr + torch, may take several minutes). Subsequent runs reuse the cached venv. Notify the user before first-time setup. +**First run** requires `uv sync` (installs funasr + torch, may take several minutes). Subsequent runs reuse the cached venv. + +**All output files (transcription JSON, clips, frames) go in the same directory as the video file.** Never use `~/whisperx_output/` or any other separate output directory — the `--output-dir` flag should point to the video's own directory. This keeps all artifacts co-located and portable. + +**Transcription can be long-running.** The agent should generate the transcription command and **give it to the user to execute manually**, rather than running it in the agent's own environment. This avoids blocking the conversation and allows the user to run it on a more powerful machine if needed. + +**If the agent detects it is running in WSL**, it should provide PowerShell commands for the user to run on the Windows host, not Linux commands. For example, convert the SKILL_DIR and VIDEO_PATH to their Windows equivalents (e.g., `C:\Users\...` instead of `/mnt/c/...`). + +Command templates (adjust paths for the user's environment): ```bash # Recommended default: Fun-ASR-Nano (high quality, Chinese-optimized) @@ -94,6 +102,8 @@ uv run --directory SKILL_DIR/scripts/transcription funasr-fast VIDEO_PATH --trac uv run --directory SKILL_DIR/scripts/transcription funasr-nano VIDEO_PATH --list-tracks ``` +Where `VIDEO_DIR` is the directory containing the video file, and `SKILL_DIR` is the skill installation directory. + **Multi-track transcription:** when the user provides multiple track indexes (e.g., track 1 = mic, track 2 = Discord), run the transcription command **once per track**. Each run produces a separate JSON file named `_track_.json`. Record each transcription in the index (Step 5) with its corresponding track index. When answering user questions, the agent should cross-reference all available transcripts to understand the full conversation context. **Skip if cached:** check the index file (Step 5) for existing transcription entries. If a record exists for the same track index and the video file hasn't changed, reuse it. @@ -325,13 +335,13 @@ Some material sites may block automated access (anti-bot, Cloudflare, CAPTCHA). Before any operation expected to take >1 minute (first-time `uv sync`, transcription of long videos, large frame extraction): -1. **Notify the user** that a long operation is about to start and give a rough time estimate. -2. Run the operation in a way that does not block the conversation. -3. Do not poll progress at high frequency — wait for completion notification rather than checking every few seconds. +1. **Generate the command and give it to the user to execute manually**, rather than running it in the agent's own environment. This avoids blocking the conversation and lets the user run it on a more powerful machine (e.g., a Windows host while the agent runs in WSL). +2. If the agent runs in WSL, provide PowerShell commands with Windows paths (e.g., `C:\Users\...` instead of `/mnt/c/...`) for the user to run on the Windows host. +3. After the user reports completion, the agent should verify the output exists before proceeding. ## Self-contained setup -The skill bundles its own transcription project under `scripts/transcription/`. On first use: +The skill bundles its own transcription project under `scripts/transcription/`. On first use, the user needs to run: ```bash cd SKILL_DIR/scripts/transcription && uv sync @@ -339,6 +349,8 @@ cd SKILL_DIR/scripts/transcription && uv sync This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user has previously installed these packages via uv in other projects, uv's cache will reuse them — first-time cost is only the link step. +If the agent runs in WSL, it should provide the equivalent PowerShell command with Windows paths for the user to run on the Windows host. + ## Common pitfalls 1. **Installing dependencies without consent.** Always ask the user first. If they decline, stop — the skill cannot proceed without ffmpeg, uv, and python3. @@ -351,6 +363,8 @@ This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user 8. **Hardcoding tool names.** This skill is agent-agnostic. Do not assume specific MCP tools or APIs exist. Use whatever the runtime provides for vision analysis, background execution, and user notification. 9. **Hardcoding output language.** The edit plan table and all user-facing text must follow the user's language, not a fixed language. The English table in Step 8 is a template — translate columns and content to match the user's language at runtime. 10. **Dropping audio tracks during clip extraction.** The default clip command maps only `0:v:0` (video-only) for fast frame extraction. If the user plans to import the clip into a video editor, use `--all-streams` (or `-map 0`) to preserve all audio tracks. Always ask the user which they need. +11. **Using `~/whisperx_output/` or other separate output directories.** All artifacts (transcription JSON, clips, frames, index file) must go in the same directory as the video file. The `--output-dir` flag should always point to the video's own directory. This keeps everything co-located and portable. +12. **Running transcription commands in the agent's own environment.** Transcription is long-running and resource-intensive. Generate the command and give it to the user to execute on their host machine. If the agent runs in WSL, provide PowerShell commands with Windows paths. ## Verification checklist