nite/video-edit-planner-skill
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: distinguish setup vs long transcription handoff

Browse Source 
- Match command syntax to execution environment: PowerShell for Windows, shell for WSL/Linux
- Choose execution by video locality, not merely where the agent runs
- Clarify that agents should perform lightweight setup themselves when possible
- Document WSL agents can use powershell.exe to run Windows host setup
- Hand off only long-running transcription/media commands to the user
This commit is contained in:
nite
2026-07-14 14:14:50 +08:00
parent 279d0a85d8
commit 0005ba554e
2 changed files with 89 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SKILL.md
+29 -26
View File
@@ -97,17 +97,21 @@ Use the bundled `funasr-script` in `scripts/transcription/`. **Prefer Fun-ASR-Na
**All output files (transcription JSON, clips, frames) go in the same directory as the video file.** Never use `~/whisperx_output/` or any other separate output directory — the `--output-dir` flag should point to the video's own directory. This keeps all artifacts co-located and portable.
**Transcription can be long-running.** The agent should generate the transcription command and **give it to the user to execute manually**, rather than running it in the agent's own environment. This avoids blocking the conversation and allows the user to run it on a more powerful machine if needed.
**Choose the execution environment based on where the video file lives and where the command will run.** Keep heavy media I/O local to the filesystem that stores the video:
**Critical: when you tell the user you will provide a command, include it immediately in the same response.** Do not say "I'll give you the transcription command" and then move on to other topics without actually printing the command. The user should never have to ask "you said you'd give me the command but didn't."
**Choose the execution environment based on where the video file lives, not based only on where the agent runs.** Keep heavy media I/O local to the filesystem that stores the video:
- If the video is on the Windows host filesystem (for example a WSL path under `/mnt/c/...`, `/mnt/d/...`, etc.), prefer running transcription on the Windows host with PowerShell and Windows paths (`C:\...`, `D:\...`). This avoids routing large video reads through WSL and usually gives better hardware/GPU access.
- If the video is inside the WSL/Linux filesystem (for example `/home/...`, `/var/...`, or another Linux mount), prefer running transcription in WSL/Linux with Linux paths. Do not force Windows just because the agent itself is running in WSL.
- If the video is on the Windows host filesystem (for example a WSL path under `/mnt/c/...`, `/mnt/d/...`, or a native Windows path like `C:\\...`, `D:\\...`), prefer running transcription on the Windows host. Give **PowerShell commands** with Windows paths.
- If the video is inside the WSL/Linux filesystem (for example `/home/...`, `/var/...`, or another Linux mount), prefer running transcription in WSL/Linux. Give **Linux shell commands** with Linux paths.
- If the location is ambiguous, ask the user where the video is stored and where they want the workload to run.
**Windows `uv` constraint:** Windows `uv` should not operate on WSL UNC paths like `\\wsl.localhost\...` or `\\wsl$\...`; it can fail while managing `.venv` directories. Therefore, when the chosen execution environment is Windows, the transcription project itself must also be available under a Windows-local path. Follow `references/wsl-windows-uv-transcription.md`: either clone the skill repo on Windows, or copy `scripts/transcription/` to a Windows-local directory, then run `uv` from that local Windows path.
**Agent should do the lightweight setup work.** Installing/checking `uv`, copying `scripts/transcription/` to a Windows-local directory, and running `uv sync` are setup tasks. If the agent can execute them through its available command interface, it should do them itself after getting any needed consent for dependency installation. Only the actual long-running transcription command should be handed to the user to execute manually.
**If the agent runs in WSL and Windows execution is chosen**, the agent can invoke the Windows host via `powershell.exe`. It should copy or clone the transcription project to a Windows-local path (for example `C:\\Users\\<user>\\AppData\\Local\\Temp\\vedit-transcription`) and run setup there. Do not ask the user to manually copy files or run `uv sync` unless the agent cannot access the host command line or lacks permission.
**Windows `uv` constraint:** Windows `uv` should not operate on WSL UNC paths like `\\\\wsl.localhost\\...` or `\\\\wsl$\\...`; it can fail while managing `.venv` directories. Therefore, when the chosen execution environment is Windows, the transcription project itself must also be available under a Windows-local path. Follow `references/wsl-windows-uv-transcription.md`.
**Transcription can be long-running.** After setup is complete, generate the final transcription command and give it to the user to execute manually. Use PowerShell syntax for Windows execution and Linux shell syntax for WSL/Linux execution.
**Critical: when you tell the user you will provide a command, include it immediately in the same response.** Do not say "I'll give you the transcription command" and then move on to other topics without actually printing the command. The user should never have to ask "you said you'd give me the command but didn't."
Command templates (adjust paths for the user's environment):
@@ -365,37 +369,36 @@ Some material sites may block automated access (anti-bot, Cloudflare, CAPTCHA).
## Long-running operations
Before any operation expected to take >1 minute (first-time `uv sync`, transcription of long videos, large frame extraction):
Before any operation expected to take >1 minute (transcription of long videos, large frame extraction):
1. **Generate the command and give it to the user to execute manually**, rather than running it in the agent's own environment. This avoids blocking the conversation and lets the user run it where the media I/O is fastest.
2. Choose PowerShell/Windows commands when the video file is on the Windows host filesystem (`/mnt/c/...`, `/mnt/d/...`, or a Windows path like `C:\...`). Choose Linux/WSL commands when the video file is stored in the WSL/Linux filesystem (`/home/...`, `/var/...`, etc.).
3. Do not point Windows `uv` at `\\wsl.localhost\...` or `\\wsl$\...` paths. If Windows execution is chosen, put/clone/copy the transcription project under a Windows local directory first, then run `uv` there.
4. After the user reports completion, the agent should verify the output exists before proceeding.
1. Do lightweight setup directly when possible: dependency checks, copying scripts, preparing a Windows-local transcription directory, and running `uv sync`.
2. Ask for consent before installing missing system dependencies (`ffmpeg`, `uv`, Python), then install them if the runtime can do so.
3. Hand off only genuinely long-running media operations to the user. Generate the command and give it to the user to execute manually.
4. Match command syntax to the chosen execution environment: PowerShell commands for Windows execution; Linux shell commands for WSL/Linux execution.
5. After the user reports completion, the agent should verify the output exists before proceeding.
## Self-contained setup
The skill bundles its own transcription project under `scripts/transcription/`. On first use, the user needs to run:
The skill bundles its own transcription project under `scripts/transcription/`. On first use, the agent should prepare the environment before handing off the transcription command.
For WSL/Linux execution, run setup in the bundled transcription directory:
```bash
cd SKILL_DIR/scripts/transcription && uv sync
```
This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user has previously installed these packages via uv in other projects, uv's cache will reuse them — first-time cost is only the link step.
If Windows execution is chosen because the video is on the Windows host filesystem, and the skill is installed under the WSL filesystem, do **not** run Windows `uv` against the WSL path. Instead, tell the user to prepare a Windows-local copy of the transcription project first:
For Windows execution, ensure the transcription project is in a Windows-local directory before running setup. If the agent itself is running in WSL, it can use `powershell.exe` to run host commands and should do this setup itself when possible:
```powershell
# Option A: clone the full skill repo on Windows (recommended for repeated use)
git clone https://git.nite07.com/nite/video-edit-planner-skill.git "$env:USERPROFILE\\video-edit-planner-skill"
Set-Location "$env:USERPROFILE\\video-edit-planner-skill\\scripts\\transcription"
uv sync
# Option B: after the agent copies scripts/transcription to a Windows-local directory (one-off use)
Set-Location "C:\\Users\\<user>\\AppData\\Local\\Temp\\vedit-transcription"
# Example Windows-local setup directory
$TransDir = "$env:LOCALAPPDATA\\Temp\\vedit-transcription"
New-Item -ItemType Directory -Force -Path $TransDir | Out-Null
# Copy or clone scripts/transcription into $TransDir before this step.
Set-Location $TransDir
uv sync
```
Subsequent Windows transcription commands should also use that same Windows-local transcription directory. If the video is stored inside WSL/Linux, run `uv sync` and transcription in WSL instead.
This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user has previously installed these packages via uv in other projects, uv's cache will reuse them — first-time cost is often only the link step. Do not ask the user to manually copy files or run `uv sync` unless the agent cannot access the host command line or lacks permission.
## Common pitfalls
@@ -410,7 +413,7 @@ Subsequent Windows transcription commands should also use that same Windows-loca
9. **Hardcoding output language.** The edit plan table and all user-facing text must follow the user's language, not a fixed language. The English table in Step 8 is a template — translate columns and content to match the user's language at runtime.
10. **Dropping audio tracks during clip extraction.** The default clip command maps only `0:v:0` (video-only) for fast frame extraction. If the user plans to import the clip into a video editor, use `--all-streams` (or `-map 0`) to preserve all audio tracks. Always ask the user which they need.
11. **Using `~/whisperx_output/` or other separate output directories.** All artifacts (transcription JSON, clips, frames, index file) must go in the same directory as the video file. The `--output-dir` flag should always point to the video's own directory. This keeps everything co-located and portable.
12. **Choosing execution environment solely from where the agent runs.** Choose based on where the video file lives. Windows-host video (`/mnt/c`, `/mnt/d`, `C:\...`, `D:\...`) → PowerShell/Windows-local transcription project. WSL/Linux video (`/home/...`, `/var/...`) → WSL/Linux commands. Never ask Windows `uv` to run from `\\wsl.localhost\...` / `\\wsl$\...`; clone or copy the transcription project to `C:\...` first when Windows execution is chosen.
12. **Choosing command syntax or execution environment solely from where the agent runs.** Choose based on where the video file lives and where the command will execute. Windows execution → PowerShell commands and Windows paths; WSL/Linux execution → Linux shell commands and Linux paths. If the agent runs in WSL but Windows execution is chosen, use `powershell.exe` to do lightweight setup (copy files, install/check dependencies, run `uv sync`) before handing only the long transcription command to the user. Never ask Windows `uv` to run from `\\\\wsl.localhost\\...` / `\\\\wsl$\\...`.
13. **Saying you'll provide a command but not including it.** If you mention giving the user a transcription command, include it immediately in that same response. Do not move on to other content without printing the actual command. The user should never have to ask for it.
14. **Sending a wall of free-text questions.** When gathering editing requirements, use structured one-at-a-time multiple-choice questions via the runtime's clarify/prompt tool. This is far more effective than dumping 10 questions as a paragraph.