From 4181323184918eb715e99e99e30135a47c1498f0 Mon Sep 17 00:00:00 2001 From: nite Date: Tue, 14 Jul 2026 17:45:10 +0800 Subject: [PATCH] docs: remove unimplemented FunASR context mode --- SKILL.md | 1 - references/funasr-nano-rolling-context.md | 104 ---------------------- 2 files changed, 105 deletions(-) delete mode 100644 references/funasr-nano-rolling-context.md diff --git a/SKILL.md b/SKILL.md index f075b82..2450cff 100644 --- a/SKILL.md +++ b/SKILL.md @@ -132,7 +132,6 @@ uv run --directory SKILL_DIR/scripts/transcription funasr-nano VIDEO_PATH --list Where `VIDEO_DIR` is the directory containing the video file, `SKILL_DIR` is the skill installation directory, and `AUDIO_LANGUAGE_CODE` is the explicit transcription language (for example `zh` for Mandarin Chinese). -**Rolling context / semantic correction:** Fun-ASR-Nano has upstream support for `prev_text`-style rolling context in streaming/incremental examples: earlier stabilized text can be fed back as context for later decoding, similar to a 2-pass-style refinement. However, do not assume the current bundled offline script automatically enables whole-file rolling semantic correction: it currently calls ordinary `AutoModel.generate(...)` on the extracted WAV and does not run a cumulative chunked pass with `prev_text`. See `references/funasr-nano-rolling-context.md`. Treat the current transcript as a strong baseline, not the final word when unclear speech or homophones matter. **Common OBS multi-track audio layouts.** When the user says "multi-track" or mentions OBS recording, these are the typical stream layouts: diff --git a/references/funasr-nano-rolling-context.md b/references/funasr-nano-rolling-context.md deleted file mode 100644 index a4d15b0..0000000 --- a/references/funasr-nano-rolling-context.md +++ /dev/null @@ -1,104 +0,0 @@ -# Fun-ASR-Nano rolling context / `prev_text` notes - -Use this when deciding whether Fun-ASR-Nano can use already-transcribed context from the same audio file to improve later sentence recognition. - -## User intent - -This is **not** about manually adding fixed hotwords. The desired behavior is closer to rolling context or 2-pass-style refinement: - -- While processing one independent audio/video file, the model should use text already recognized from earlier audio as context for later audio. -- When a current sentence is ambiguous because of unclear pronunciation, it may benefit from preceding vocabulary, phrasing, topic, and dialogue context. -- This is analogous to streaming ASR systems where early partial text may be wrong, and a later stabilized/final pass revises the sentence with more context. - -## Verified upstream behavior - -Sources checked: - -- `modelscope/FunASR` `funasr/models/fun_asr_nano/model.py` -- `modelscope/FunASR` `examples/industrial_data_pretraining/fun_asr_nano/demo2.py` -- `modelscope/FunASR` `funasr/models/fun_asr_nano/inference_vllm_streaming.py` -- `modelscope/FunASR` `docs/vllm_guide.md` -- `modelscope/FunASR` `docs/vllm_guide_zh.md` - -Fun-ASR-Nano supports a `prev_text` mechanism in the model inference path. In `model.py`, `prev_text` is appended into the assistant-side prompt prefix before decoding: - -```python -if kwargs.get("prev_text", None) is not None: - source_input += kwargs["prev_text"] -``` - -The official `demo2.py` demonstrates cumulative chunk transcription: - -```python -prev_text = "" -for idx, cum_duration in enumerate(cum_durations): - audio, rate = load_audio(wav_path, 16000, duration=round(cum_duration, 3)) - prev_text = m.inference([torch.tensor(audio)], prev_text=prev_text, **kwargs)[0][0]["text"] - if idx != len(cum_durations) - 1: - prev_text = tokenizer.decode(tokenizer.encode(prev_text)[:-5]).replace("�", "") -``` - -The streaming vLLM code also uses a two-stage design: - -- Stage 1: generate early chunks without `prev_text` to find stable output. -- Stage 2: use stable output as `prev_text` for later cumulative chunks. -- It keeps a rollback/unfixed tail, so the last few characters may change as more audio arrives. - -The vLLM guide explicitly describes this as: - -- first chunks: no `prev_text` -- subsequent chunks: use stable output as `prev_text` - -## What the current bundled script does - -The current `scripts/transcription/funasr_common.py` path calls ordinary offline `AutoModel.generate(input=..., batch_size=1)` on the extracted WAV file. - -It currently passes: - -- `input` -- `batch_size=1` -- explicit `language` when non-`auto` -- `use_itn=True` for Fun-ASR-Nano - -It does **not** currently implement a chunked/cumulative pass that feeds prior recognized text back through `prev_text`. - -Therefore: - -- Fun-ASR-Nano has model-level support for rolling text context. -- Upstream examples show this mechanism in streaming/incremental usage. -- But the current bundled offline script should **not** be described as automatically enabling whole-file rolling context correction. - -## Practical implication for this skill - -When accuracy matters, especially for unclear speech, homophones, game terms, and dialogue continuity, do not assume the default offline transcript is the best possible use of Nano's context mechanism. - -The skill should describe this honestly: - -1. Current default offline transcription is a strong baseline. -2. It may still produce wrong words such as `帐篷` -> `账本` when source speech is unclear. -3. Fun-ASR-Nano has `prev_text`/rolling-context mechanisms upstream, but the bundled script does not yet expose a stable offline mode for that. -4. A future enhancement can evaluate a chunked/cumulative or 2-pass-like transcription mode that feeds stabilized earlier text as `prev_text`. -5. Such a mode should be benchmarked before becoming default, because cumulative re-encoding may be slower and may have different timestamp/segmentation behavior. - -## Future enhancement idea - -A possible experimental mode: - -```bash -funasr-nano VIDEO_PATH \ - --track TRACK_INDEX \ - --language AUDIO_LANGUAGE_CODE \ - --context-mode rolling \ - --rollback-chars 5 \ - --output-dir VIDEO_DIR -``` - -Implementation direction: - -- Split or cumulatively extend audio in chunks. -- Keep a stable prefix and an unfixed tail, similar to upstream `demo2.py` / streaming vLLM logic. -- Feed the stable prefix into `prev_text` for the next pass. -- Preserve a normal offline mode because it is simpler, faster, and already works. -- Compare quality on real user samples before enabling by default. - -This is a transcription-mode experiment, not a hotword-list feature.