docs: document long outputs and FunASR hotwords
This commit is contained in:
@@ -89,6 +89,8 @@ If the agent's runtime supports structured questioning (e.g., a `clarify` tool o
|
||||
|
||||
**Completion criteria:** user has described at least a rough editing goal, and the agent has probed the relevant axes above.
|
||||
|
||||
**Long requirement summaries:** if the synthesized requirements, transcript-derived notes, or edit-planning response will be long, do not dump the whole thing into chat by default. Ask the user whether to save it as a file instead. Markdown (`.md`) is the default format for requirement summaries and edit plans; if the user requests another format (JSON, CSV, DOCX, etc.), use the requested format when practical.
|
||||
|
||||
### Step 4 — Transcribe audio
|
||||
|
||||
Use the bundled `funasr-script` in `scripts/transcription/`. **Prefer Fun-ASR-Nano** as the default transcription model — it has higher accuracy and sensitivity on Chinese audio. Use SenseVoice only for fast previews.
|
||||
@@ -130,6 +132,8 @@ uv run --directory SKILL_DIR/scripts/transcription funasr-nano VIDEO_PATH --list
|
||||
|
||||
Where `VIDEO_DIR` is the directory containing the video file, `SKILL_DIR` is the skill installation directory, and `AUDIO_LANGUAGE_CODE` is the explicit transcription language (for example `zh` for Mandarin Chinese).
|
||||
|
||||
**Context and hotwords:** Fun-ASR-Nano is an LLM-based ASR model and upstream code supports prompt-style `language` and `hotwords` hints. However, do not assume the current bundled script automatically performs cross-segment semantic correction: it currently passes `language` / `use_itn`, but does not pass external hotwords or rolling `prev_text` context between VAD segments. If the video contains known game terms, names, locations, item names, or other jargon, ask the user for a hotword list or derive candidate terms and confirm them before transcription. See `references/funasr-nano-context-hotwords.md`. Hotwords are biasing hints, not a substitute for final manual checking.
|
||||
|
||||
**Common OBS multi-track audio layouts.** When the user says "multi-track" or mentions OBS recording, these are the typical stream layouts:
|
||||
|
||||
| Track | Typical content |
|
||||
@@ -319,7 +323,7 @@ For fast-paced story + comedy gameplay videos, cull hard: keep combat, danger, r
|
||||
|
||||
This workflow is especially effective for requests like **“find fast-paced funny + story-driven candidate segments”**. It keeps token/runtime cost low while still verifying whether a transcript-highlighted moment actually has usable on-screen action.
|
||||
|
||||
Output a Markdown table with overall recommendations, then let the user iterate. **Respond in the same language the user is using.**
|
||||
Output a Markdown table with overall recommendations, then let the user iterate. **Respond in the same language the user is using.** If the plan is likely to be long (large tables, detailed scripts, full requirement summaries, or multi-section recommendations), ask whether the user wants it saved as a file instead of pasted into chat. Default to Markdown (`.md`) for saved plans unless the user requests another format.
|
||||
|
||||
```markdown
|
||||
## Edit Plan
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
# Fun-ASR-Nano context, hotwords, and transcript correction notes
|
||||
|
||||
Use this when deciding whether Fun-ASR-Nano can use semantic context to improve recognition quality.
|
||||
|
||||
## Verified from upstream code/docs
|
||||
|
||||
Sources checked:
|
||||
|
||||
- `modelscope/FunASR` `funasr/models/fun_asr_nano/model.py`
|
||||
- `modelscope/FunASR` `funasr/models/fun_asr_nano/inference_vllm.py`
|
||||
- `modelscope/FunASR` `funasr/models/fun_asr_nano/inference_vllm_pipeline.py`
|
||||
- `modelscope/FunASR` `funasr/models/fun_asr_nano/inference_vllm_streaming.py`
|
||||
- `modelscope/FunASR` `docs/vllm_guide_zh.md`
|
||||
- `modelscope/FunASR` `docs/model_selection.md`
|
||||
|
||||
The upstream model is LLM-based: a SenseVoice-style audio encoder plus an LLM decoder. The upstream model-selection guide describes Fun-ASR-Nano as stronger on hard cases, context, and proper nouns.
|
||||
|
||||
The PyTorch path has a prompt builder roughly shaped as:
|
||||
|
||||
```python
|
||||
def get_prompt(self, hotwords: list[str], language: str = None, itn: bool = True):
|
||||
if len(hotwords) > 0:
|
||||
prompt = "请结合上下文信息，更加准确地完成语音转写任务。如果没有相关信息，我们会留空。\n\n\n**上下文信息：**\n\n\n"
|
||||
prompt += f"热词列表：[{hotwords}]\n"
|
||||
else:
|
||||
prompt = ""
|
||||
if language is None:
|
||||
prompt += "语音转写"
|
||||
else:
|
||||
prompt += f"语音转写成{language}"
|
||||
if not itn:
|
||||
prompt += "，不进行文本规整"
|
||||
return prompt + "："
|
||||
```
|
||||
|
||||
The vLLM path exposes the same idea via `hotwords` and `language`. The vLLM guide documents `language` and `hotwords` as `generate()` / API parameters.
|
||||
|
||||
## What is enabled by default?
|
||||
|
||||
Do **not** assume cross-segment semantic correction is automatically enabled for offline file transcription.
|
||||
|
||||
In the bundled transcription script at the time of writing:
|
||||
|
||||
- `funasr_common.py` calls `model.generate(input=..., batch_size=1)`.
|
||||
- It passes `language` only when the user specified a non-`auto` language.
|
||||
- It passes `use_itn=True` for Fun-ASR-Nano.
|
||||
- It does **not** pass `hotwords`.
|
||||
- It does **not** pass `prev_text` or a rolling context between VAD segments.
|
||||
|
||||
Therefore, the current script uses Nano's model-internal contextual ability, language hint, and ITN, but it does not provide external domain context or previous transcript text. The transcript may still contain homophone/near-homophone mistakes such as `帐篷` -> `账本` when the source audio is unclear.
|
||||
|
||||
## Practical guidance
|
||||
|
||||
1. Use explicit `--language AUDIO_LANGUAGE_CODE` to avoid language ambiguity.
|
||||
2. If the content has known names, game terms, locations, item names, or recurring jargon, collect them as a hotword list before transcription.
|
||||
3. Prefer adding explicit script support such as `--hotword TERM` / `--hotwords-file FILE` over relying on vague “semantic correction”.
|
||||
4. Treat hotwords as biasing hints, not proof. They can improve proper nouns and domain terms but cannot fully fix unclear audio.
|
||||
5. For final captions, quotes, subtitles, narration, or published copy, still re-listen and manually correct the original audio segment.
|
||||
6. Do not silently invent hotwords. Ask the user for domain terms or extract candidate terms from existing project notes/transcripts and confirm them.
|
||||
|
||||
## Possible future script enhancement
|
||||
|
||||
Expose hotwords in the bundled CLI:
|
||||
|
||||
```bash
|
||||
funasr-nano VIDEO_PATH \
|
||||
--track TRACK_INDEX \
|
||||
--language AUDIO_LANGUAGE_CODE \
|
||||
--hotword "帐篷" \
|
||||
--hotword "复活点" \
|
||||
--output-dir VIDEO_DIR
|
||||
```
|
||||
|
||||
Then pass them to FunASR:
|
||||
|
||||
```python
|
||||
gen_kw["hotwords"] = args.hotwords
|
||||
```
|
||||
|
||||
For longer lists, support a UTF-8 text file with one hotword per line.
|
||||
Reference in New Issue
Block a user