nite/video-edit-planner-skill
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: distinguish setup vs long transcription handoff

Browse Source 
- Match command syntax to execution environment: PowerShell for Windows, shell for WSL/Linux
- Choose execution by video locality, not merely where the agent runs
- Clarify that agents should perform lightweight setup themselves when possible
- Document WSL agents can use powershell.exe to run Windows host setup
- Hand off only long-running transcription/media commands to the user
This commit is contained in:
nite
2026-07-14 14:14:50 +08:00
parent 279d0a85d8
commit 0005ba554e
2 changed files with 89 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SKILL.md
+29 -26
View File
@@ -97,17 +97,21 @@ Use the bundled `funasr-script` in `scripts/transcription/`. **Prefer Fun-ASR-Na
**All output files (transcription JSON, clips, frames) go in the same directory as the video file.** Never use `~/whisperx_output/` or any other separate output directory — the `--output-dir` flag should point to the video's own directory. This keeps all artifacts co-located and portable.
**Transcription can be long-running.** The agent should generate the transcription command and **give it to the user to execute manually**, rather than running it in the agent's own environment. This avoids blocking the conversation and allows the user to run it on a more powerful machine if needed.
**Choose the execution environment based on where the video file lives and where the command will run.** Keep heavy media I/O local to the filesystem that stores the video:
**Critical: when you tell the user you will provide a command, include it immediately in the same response.** Do not say "I'll give you the transcription command" and then move on to other topics without actually printing the command. The user should never have to ask "you said you'd give me the command but didn't."
**Choose the execution environment based on where the video file lives, not based only on where the agent runs.** Keep heavy media I/O local to the filesystem that stores the video:
- If the video is on the Windows host filesystem (for example a WSL path under `/mnt/c/...`, `/mnt/d/...`, etc.), prefer running transcription on the Windows host with PowerShell and Windows paths (`C:\...`, `D:\...`). This avoids routing large video reads through WSL and usually gives better hardware/GPU access.
- If the video is inside the WSL/Linux filesystem (for example `/home/...`, `/var/...`, or another Linux mount), prefer running transcription in WSL/Linux with Linux paths. Do not force Windows just because the agent itself is running in WSL.
- If the video is on the Windows host filesystem (for example a WSL path under `/mnt/c/...`, `/mnt/d/...`, or a native Windows path like `C:\\...`, `D:\\...`), prefer running transcription on the Windows host. Give **PowerShell commands** with Windows paths.
- If the video is inside the WSL/Linux filesystem (for example `/home/...`, `/var/...`, or another Linux mount), prefer running transcription in WSL/Linux. Give **Linux shell commands** with Linux paths.
- If the location is ambiguous, ask the user where the video is stored and where they want the workload to run.
**Windows `uv` constraint:** Windows `uv` should not operate on WSL UNC paths like `\\wsl.localhost\...` or `\\wsl$\...`; it can fail while managing `.venv` directories. Therefore, when the chosen execution environment is Windows, the transcription project itself must also be available under a Windows-local path. Follow `references/wsl-windows-uv-transcription.md`: either clone the skill repo on Windows, or copy `scripts/transcription/` to a Windows-local directory, then run `uv` from that local Windows path.
**Agent should do the lightweight setup work.** Installing/checking `uv`, copying `scripts/transcription/` to a Windows-local directory, and running `uv sync` are setup tasks. If the agent can execute them through its available command interface, it should do them itself after getting any needed consent for dependency installation. Only the actual long-running transcription command should be handed to the user to execute manually.
**If the agent runs in WSL and Windows execution is chosen**, the agent can invoke the Windows host via `powershell.exe`. It should copy or clone the transcription project to a Windows-local path (for example `C:\\Users\\<user>\\AppData\\Local\\Temp\\vedit-transcription`) and run setup there. Do not ask the user to manually copy files or run `uv sync` unless the agent cannot access the host command line or lacks permission.
**Windows `uv` constraint:** Windows `uv` should not operate on WSL UNC paths like `\\\\wsl.localhost\\...` or `\\\\wsl$\\...`; it can fail while managing `.venv` directories. Therefore, when the chosen execution environment is Windows, the transcription project itself must also be available under a Windows-local path. Follow `references/wsl-windows-uv-transcription.md`.
**Transcription can be long-running.** After setup is complete, generate the final transcription command and give it to the user to execute manually. Use PowerShell syntax for Windows execution and Linux shell syntax for WSL/Linux execution.
**Critical: when you tell the user you will provide a command, include it immediately in the same response.** Do not say "I'll give you the transcription command" and then move on to other topics without actually printing the command. The user should never have to ask "you said you'd give me the command but didn't."
Command templates (adjust paths for the user's environment):
@@ -365,37 +369,36 @@ Some material sites may block automated access (anti-bot, Cloudflare, CAPTCHA).
## Long-running operations
Before any operation expected to take >1 minute (first-time `uv sync`, transcription of long videos, large frame extraction):
Before any operation expected to take >1 minute (transcription of long videos, large frame extraction):
1. **Generate the command and give it to the user to execute manually**, rather than running it in the agent's own environment. This avoids blocking the conversation and lets the user run it where the media I/O is fastest.
2. Choose PowerShell/Windows commands when the video file is on the Windows host filesystem (`/mnt/c/...`, `/mnt/d/...`, or a Windows path like `C:\...`). Choose Linux/WSL commands when the video file is stored in the WSL/Linux filesystem (`/home/...`, `/var/...`, etc.).
3. Do not point Windows `uv` at `\\wsl.localhost\...` or `\\wsl$\...` paths. If Windows execution is chosen, put/clone/copy the transcription project under a Windows local directory first, then run `uv` there.
4. After the user reports completion, the agent should verify the output exists before proceeding.
1. Do lightweight setup directly when possible: dependency checks, copying scripts, preparing a Windows-local transcription directory, and running `uv sync`.
2. Ask for consent before installing missing system dependencies (`ffmpeg`, `uv`, Python), then install them if the runtime can do so.
3. Hand off only genuinely long-running media operations to the user. Generate the command and give it to the user to execute manually.
4. Match command syntax to the chosen execution environment: PowerShell commands for Windows execution; Linux shell commands for WSL/Linux execution.
5. After the user reports completion, the agent should verify the output exists before proceeding.
## Self-contained setup
The skill bundles its own transcription project under `scripts/transcription/`. On first use, the user needs to run:
The skill bundles its own transcription project under `scripts/transcription/`. On first use, the agent should prepare the environment before handing off the transcription command.
For WSL/Linux execution, run setup in the bundled transcription directory:
```bash
cd SKILL_DIR/scripts/transcription && uv sync
```
This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user has previously installed these packages via uv in other projects, uv's cache will reuse them — first-time cost is only the link step.
If Windows execution is chosen because the video is on the Windows host filesystem, and the skill is installed under the WSL filesystem, do **not** run Windows `uv` against the WSL path. Instead, tell the user to prepare a Windows-local copy of the transcription project first:
For Windows execution, ensure the transcription project is in a Windows-local directory before running setup. If the agent itself is running in WSL, it can use `powershell.exe` to run host commands and should do this setup itself when possible:
```powershell
# Option A: clone the full skill repo on Windows (recommended for repeated use)
git clone https://git.nite07.com/nite/video-edit-planner-skill.git "$env:USERPROFILE\\video-edit-planner-skill"
Set-Location "$env:USERPROFILE\\video-edit-planner-skill\\scripts\\transcription"
uv sync
# Option B: after the agent copies scripts/transcription to a Windows-local directory (one-off use)
Set-Location "C:\\Users\\<user>\\AppData\\Local\\Temp\\vedit-transcription"
# Example Windows-local setup directory
$TransDir = "$env:LOCALAPPDATA\\Temp\\vedit-transcription"
New-Item -ItemType Directory -Force -Path $TransDir | Out-Null
# Copy or clone scripts/transcription into $TransDir before this step.
Set-Location $TransDir
uv sync
```
Subsequent Windows transcription commands should also use that same Windows-local transcription directory. If the video is stored inside WSL/Linux, run `uv sync` and transcription in WSL instead.
This creates an isolated venv with `funasr`, `torch`, `torchaudio`. If the user has previously installed these packages via uv in other projects, uv's cache will reuse them — first-time cost is often only the link step. Do not ask the user to manually copy files or run `uv sync` unless the agent cannot access the host command line or lacks permission.
## Common pitfalls
@@ -410,7 +413,7 @@ Subsequent Windows transcription commands should also use that same Windows-loca
9. **Hardcoding output language.** The edit plan table and all user-facing text must follow the user's language, not a fixed language. The English table in Step 8 is a template — translate columns and content to match the user's language at runtime.
10. **Dropping audio tracks during clip extraction.** The default clip command maps only `0:v:0` (video-only) for fast frame extraction. If the user plans to import the clip into a video editor, use `--all-streams` (or `-map 0`) to preserve all audio tracks. Always ask the user which they need.
11. **Using `~/whisperx_output/` or other separate output directories.** All artifacts (transcription JSON, clips, frames, index file) must go in the same directory as the video file. The `--output-dir` flag should always point to the video's own directory. This keeps everything co-located and portable.
12. **Choosing execution environment solely from where the agent runs.** Choose based on where the video file lives. Windows-host video (`/mnt/c`, `/mnt/d`, `C:\...`, `D:\...`) → PowerShell/Windows-local transcription project. WSL/Linux video (`/home/...`, `/var/...`) → WSL/Linux commands. Never ask Windows `uv` to run from `\\wsl.localhost\...` / `\\wsl$\...`; clone or copy the transcription project to `C:\...` first when Windows execution is chosen.
12. **Choosing command syntax or execution environment solely from where the agent runs.** Choose based on where the video file lives and where the command will execute. Windows execution → PowerShell commands and Windows paths; WSL/Linux execution → Linux shell commands and Linux paths. If the agent runs in WSL but Windows execution is chosen, use `powershell.exe` to do lightweight setup (copy files, install/check dependencies, run `uv sync`) before handing only the long transcription command to the user. Never ask Windows `uv` to run from `\\\\wsl.localhost\\...` / `\\\\wsl$\\...`.
13. **Saying you'll provide a command but not including it.** If you mention giving the user a transcription command, include it immediately in that same response. Do not move on to other content without printing the actual command. The user should never have to ask for it.
14. **Sending a wall of free-text questions.** When gathering editing requirements, use structured one-at-a-time multiple-choice questions via the runtime's clarify/prompt tool. This is far more effective than dumping 10 questions as a paragraph.
references/wsl-windows-uv-transcription.md
+60 -29
View File
@@ -1,47 +1,70 @@
# Choosing Windows vs WSL for `uv` transcription
Use this reference when the agent may be running in WSL but the user has both a Windows host and a WSL/Linux environment.
Use this reference when the user has both a Windows host and a WSL/Linux environment.
## Core rule: follow the video file
## Core rule: follow the video file and the execution environment
Choose the execution environment based on **where the video file is stored**, not based only on where the agent runs.
Choose the execution environment based on **where the video file is stored** and therefore where heavy media I/O should happen.
| Video location | Preferred execution | Why |
|---|---|---|
| Windows host filesystem (`C:\...`, `D:\...`, or WSL paths like `/mnt/c/...`, `/mnt/d/...`) | Windows host, PowerShell, Windows paths | Keeps heavy video reads local to Windows; usually better host/GPU access |
| WSL/Linux filesystem (`/home/...`, `/var/...`, Linux project directories) | WSL/Linux shell, Linux paths | Keeps heavy video reads local to Linux; avoids Windows reading through WSL UNC |
| Ambiguous / network / external drive | Ask the user | Storage locality and GPU/tool availability may differ |
| Video location | Preferred execution | Command syntax | Why |
|---|---|---|---|
| Windows host filesystem (`C:\...`, `D:\...`, or WSL paths like `/mnt/c/...`, `/mnt/d/...`) | Windows host | PowerShell + Windows paths | Keeps heavy video reads local to Windows; usually better host/GPU access |
| WSL/Linux filesystem (`/home/...`, `/var/...`, Linux project directories) | WSL/Linux shell | Linux shell + Linux paths | Keeps heavy video reads local to Linux; avoids Windows reading through WSL UNC |
| Ambiguous / network / external drive | Ask the user | Depends on answer | Storage locality and GPU/tool availability may differ |
Do **not** assume Windows execution just because the agent itself runs in WSL. If the video is genuinely inside the WSL filesystem, running the Linux transcription project in WSL is the correct choice.
## What the agent should do vs what the user should do
The agent should do lightweight setup work when it has the capability:
- Check whether `uv`, Python, and ffmpeg are available in the chosen execution environment.
- Ask for consent before installing missing system dependencies, then install them if possible.
- If Windows execution is chosen and the agent is running in WSL, use `powershell.exe` to access the Windows host.
- Copy or clone the transcription project to a Windows-local directory when needed.
- Run `uv sync` during setup when possible.
The user should only be asked to run the **long-running transcription command** (or other long-running media processing commands), because those may take a long time and should not block the conversation.
## Windows `uv` constraint
When Windows execution is chosen, do **not** ask Windows `uv` to operate on the skill's WSL filesystem path (`\\wsl.localhost\...` / `\\wsl$\...`). Windows `uv` can fail while creating or removing `.venv` directories on UNC-backed WSL paths.
Instead, make the transcription project available under a Windows-local path and run `uv` there.
Recommended options:
### Windows setup from a WSL agent
1. **Windows-local clone** (best for repeated use)
```powershell
git clone https://git.nite07.com/nite/video-edit-planner-skill.git "$env:USERPROFILE\video-edit-planner-skill"
Set-Location "$env:USERPROFILE\video-edit-planner-skill\scripts\transcription"
uv sync
```
2. **Windows-local copy** (best for one-off use)
- Agent copies `scripts/transcription/` from the WSL skill directory to a Windows path, e.g. `/mnt/c/Users/<user>/AppData/Local/Temp/vedit-transcription/`.
- Then give the user:
```powershell
Set-Location "C:\Users\<user>\AppData\Local\Temp\vedit-transcription"
uv sync
```
If the agent runs in WSL, it can use `powershell.exe` to run Windows host setup commands. Example pattern:
## Windows command shape
```bash
# From WSL: copy bundled transcription project to a Windows-local path.
SRC="$HOME/.hermes/skills/media/video-edit-planner/scripts/transcription"
DST="/mnt/c/Users/<user>/AppData/Local/Temp/vedit-transcription"
mkdir -p "$DST"
cp -a "$SRC"/. "$DST"/
Use this when the video is stored on the Windows host filesystem:
# From WSL: run Windows uv setup through PowerShell.
powershell.exe -NoProfile -Command '
$TransDir = "$env:LOCALAPPDATA\Temp\vedit-transcription"
Set-Location $TransDir
uv sync
'
```
If copying from WSL is not possible, clone the full repo on Windows instead:
```powershell
$SkillTranscription = "$env:USERPROFILE\video-edit-planner-skill\scripts\transcription"
git clone https://git.nite07.com/nite/video-edit-planner-skill.git "$env:USERPROFILE\video-edit-planner-skill"
Set-Location "$env:USERPROFILE\video-edit-planner-skill\scripts\transcription"
uv sync
```
## Windows transcription command shape
Use this when the video is stored on the Windows host filesystem. Give this kind of command to the user after setup is complete:
```powershell
$SkillTranscription = "$env:LOCALAPPDATA\Temp\vedit-transcription"
$Video = "D:\Videos\example.mkv"
$VideoDir = Split-Path -Parent $Video
uv run --directory $SkillTranscription funasr-nano $Video --track 2 --output-dir $VideoDir
@@ -49,9 +72,16 @@ uv run --directory $SkillTranscription funasr-nano $Video --track 2 --output-dir
For multiple tracks, run once per track and record each resulting JSON in the index.
## WSL/Linux command shape
## WSL/Linux setup and transcription command shape
Use this when the video is stored inside the WSL/Linux filesystem:
Use this when the video is stored inside the WSL/Linux filesystem. The agent can run setup directly in WSL:
```bash
SKILL_TRANSCRIPTION="$HOME/.hermes/skills/media/video-edit-planner/scripts/transcription"
cd "$SKILL_TRANSCRIPTION" && uv sync
```
Then give the user a Linux shell command for the long transcription step:
```bash
SKILL_TRANSCRIPTION="$HOME/.hermes/skills/media/video-edit-planner/scripts/transcription"
@@ -65,7 +95,8 @@ If the user installed the skill somewhere else, adjust `SKILL_TRANSCRIPTION` acc
## Notes for agents
- Prefer Fun-ASR-Nano unless the user explicitly wants a fast preview.
- Give the command immediately when promising a command.
- Give the transcription command immediately when promising a command.
- Keep all outputs next to the video file.
- Avoid separate output directories such as `~/whisperx_output/`; transcription JSON, clips, frames, and the index should stay with the video.
- Use Windows paths (`C:\...`, `D:\...`) only for Windows execution; use Linux paths for WSL execution.
- Use PowerShell commands for Windows execution; use Linux shell commands for WSL/Linux execution.
- Do lightweight setup yourself when possible; hand off only long-running media processing to the user.