diff --git a/SKILL.md b/SKILL.md index f364871..9a6d2a0 100644 --- a/SKILL.md +++ b/SKILL.md @@ -89,6 +89,8 @@ If the agent's runtime supports structured questioning (e.g., a `clarify` tool o **Completion criteria:** user has described at least a rough editing goal, and the agent has probed the relevant axes above. +**Long requirement summaries:** if the synthesized requirements, transcript-derived notes, or edit-planning response will be long, do not dump the whole thing into chat by default. Ask the user whether to save it as a file instead. Markdown (`.md`) is the default format for requirement summaries and edit plans; if the user requests another format (JSON, CSV, DOCX, etc.), use the requested format when practical. + ### Step 4 — Transcribe audio Use the bundled `funasr-script` in `scripts/transcription/`. **Prefer Fun-ASR-Nano** as the default transcription model — it has higher accuracy and sensitivity on Chinese audio. Use SenseVoice only for fast previews. @@ -130,6 +132,8 @@ uv run --directory SKILL_DIR/scripts/transcription funasr-nano VIDEO_PATH --list Where `VIDEO_DIR` is the directory containing the video file, `SKILL_DIR` is the skill installation directory, and `AUDIO_LANGUAGE_CODE` is the explicit transcription language (for example `zh` for Mandarin Chinese). +**Context and hotwords:** Fun-ASR-Nano is an LLM-based ASR model and upstream code supports prompt-style `language` and `hotwords` hints. However, do not assume the current bundled script automatically performs cross-segment semantic correction: it currently passes `language` / `use_itn`, but does not pass external hotwords or rolling `prev_text` context between VAD segments. If the video contains known game terms, names, locations, item names, or other jargon, ask the user for a hotword list or derive candidate terms and confirm them before transcription. See `references/funasr-nano-context-hotwords.md`. Hotwords are biasing hints, not a substitute for final manual checking. + **Common OBS multi-track audio layouts.** When the user says "multi-track" or mentions OBS recording, these are the typical stream layouts: | Track | Typical content | @@ -319,7 +323,7 @@ For fast-paced story + comedy gameplay videos, cull hard: keep combat, danger, r This workflow is especially effective for requests like **“find fast-paced funny + story-driven candidate segments”**. It keeps token/runtime cost low while still verifying whether a transcript-highlighted moment actually has usable on-screen action. -Output a Markdown table with overall recommendations, then let the user iterate. **Respond in the same language the user is using.** +Output a Markdown table with overall recommendations, then let the user iterate. **Respond in the same language the user is using.** If the plan is likely to be long (large tables, detailed scripts, full requirement summaries, or multi-section recommendations), ask whether the user wants it saved as a file instead of pasted into chat. Default to Markdown (`.md`) for saved plans unless the user requests another format. ```markdown ## Edit Plan diff --git a/references/funasr-nano-context-hotwords.md b/references/funasr-nano-context-hotwords.md new file mode 100644 index 0000000..82ad4b5 --- /dev/null +++ b/references/funasr-nano-context-hotwords.md @@ -0,0 +1,80 @@ +# Fun-ASR-Nano context, hotwords, and transcript correction notes + +Use this when deciding whether Fun-ASR-Nano can use semantic context to improve recognition quality. + +## Verified from upstream code/docs + +Sources checked: + +- `modelscope/FunASR` `funasr/models/fun_asr_nano/model.py` +- `modelscope/FunASR` `funasr/models/fun_asr_nano/inference_vllm.py` +- `modelscope/FunASR` `funasr/models/fun_asr_nano/inference_vllm_pipeline.py` +- `modelscope/FunASR` `funasr/models/fun_asr_nano/inference_vllm_streaming.py` +- `modelscope/FunASR` `docs/vllm_guide_zh.md` +- `modelscope/FunASR` `docs/model_selection.md` + +The upstream model is LLM-based: a SenseVoice-style audio encoder plus an LLM decoder. The upstream model-selection guide describes Fun-ASR-Nano as stronger on hard cases, context, and proper nouns. + +The PyTorch path has a prompt builder roughly shaped as: + +```python +def get_prompt(self, hotwords: list[str], language: str = None, itn: bool = True): + if len(hotwords) > 0: + prompt = "请结合上下文信息,更加准确地完成语音转写任务。如果没有相关信息,我们会留空。\n\n\n**上下文信息:**\n\n\n" + prompt += f"热词列表:[{hotwords}]\n" + else: + prompt = "" + if language is None: + prompt += "语音转写" + else: + prompt += f"语音转写成{language}" + if not itn: + prompt += ",不进行文本规整" + return prompt + ":" +``` + +The vLLM path exposes the same idea via `hotwords` and `language`. The vLLM guide documents `language` and `hotwords` as `generate()` / API parameters. + +## What is enabled by default? + +Do **not** assume cross-segment semantic correction is automatically enabled for offline file transcription. + +In the bundled transcription script at the time of writing: + +- `funasr_common.py` calls `model.generate(input=..., batch_size=1)`. +- It passes `language` only when the user specified a non-`auto` language. +- It passes `use_itn=True` for Fun-ASR-Nano. +- It does **not** pass `hotwords`. +- It does **not** pass `prev_text` or a rolling context between VAD segments. + +Therefore, the current script uses Nano's model-internal contextual ability, language hint, and ITN, but it does not provide external domain context or previous transcript text. The transcript may still contain homophone/near-homophone mistakes such as `帐篷` -> `账本` when the source audio is unclear. + +## Practical guidance + +1. Use explicit `--language AUDIO_LANGUAGE_CODE` to avoid language ambiguity. +2. If the content has known names, game terms, locations, item names, or recurring jargon, collect them as a hotword list before transcription. +3. Prefer adding explicit script support such as `--hotword TERM` / `--hotwords-file FILE` over relying on vague “semantic correction”. +4. Treat hotwords as biasing hints, not proof. They can improve proper nouns and domain terms but cannot fully fix unclear audio. +5. For final captions, quotes, subtitles, narration, or published copy, still re-listen and manually correct the original audio segment. +6. Do not silently invent hotwords. Ask the user for domain terms or extract candidate terms from existing project notes/transcripts and confirm them. + +## Possible future script enhancement + +Expose hotwords in the bundled CLI: + +```bash +funasr-nano VIDEO_PATH \ + --track TRACK_INDEX \ + --language AUDIO_LANGUAGE_CODE \ + --hotword "帐篷" \ + --hotword "复活点" \ + --output-dir VIDEO_DIR +``` + +Then pass them to FunASR: + +```python +gen_kw["hotwords"] = args.hotwords +``` + +For longer lists, support a UTF-8 text file with one hotword per line.