LLM-based architectures offer strong generalization and instruction-following capabilities for streaming automatic speech recognition (ASR), but multi-talker use introduces coupled transcription, attribution, and latency challenges. Our wearable setting requires real-time transcription of a conversation between a near-field device wearer and a far-field partner. Because partial hypotheses are rendered on persistent sides of a dialogue UI, speaker roles cannot be permuted and each speaker tag must precede its turn. These sparse tags are weakly optimized by supervised fine-tuning (SFT), while committing to a leading tag can delay the text that follows. We therefore post-train a streaming LLM-based ASR model with reinforcement learning with verifiable rewards (RLVR), combining a fixed-role Speaker-Attributed WER (SA-WER) grader with a turn-level onset-delay reward. On an eight-direction multi-talker cohort, SA-WER RL reduces SA-WER from $20.51\%$ for SFT to $13.14\%$. Adding the delay objective reduces onset $p50$ by $65\%$ (from $0.918\,\text{s}$ to $0.320\,\text{s}$) and $p90$ by $44\%$ (from $1.389\,\text{s}$ to $0.780\,\text{s}$) relative to SA-WER-only RL, while achieving $12.70\%$ SA-WER. These results demonstrate that fixed-role recognition quality and streaming responsiveness can be improved jointly.
不对,等下,有没有需要分小节的?比如,原文没有小节标题,我们不能自己加h2/h3,因为那样会增加内容,违反不得增删改的规则。对,因为h2/h3是标题,原文没有,所以不能加。那整个就是一个p,里面重要的部分用strong。等下,再检查有没有其他需要处理的,比如公式,保留原样就好,用户要求保留原文所有信息,所以$20.51\%$这些直接保留。对。那这样的话,输出就是这个p标签包裹的内容,里面的关键句子加strong。等下,再确认规则:只输出HTML片段,不要其他,对。那现在整理一下: 首先,开头的背景句是重要的,加strong。然后实验结果的核心句子加strong,最后结论加strong。对吗?或者有没有其他重要的?比如方法部分?比如We therefore post-train...那个句子要不要加?也可以?不过不要太多。等下,再想,原文的结构: 1. 背景与问题:LLM-based...delay the text that follows. 2. 方法:We therefore...onset-delay reward. 3. 实验结果:On an eight-direction...12.70% SA-WER. 4. 结论:These results...jointly. 那是不是可以把这些作为不同的段落?不对,原文是连续的,用户给的正文没有分段,所以我们不能自己拆分段落,因为那样会改变原文的分段?不对,等下,用户说“每个自然段落用包裹”,原文的自然段落就是一个,所以用一个p。然后重要的句子用strong。那现在写出来:
LLM-based architectures offer strong generalization and instruction-following capabilities for streaming automatic speech recognition (ASR), but multi-talker use introduces coupled transcription, attribution, and latency challenges. Our wearable setting requires real-time transcription of a conversation between a near-field device wearer and a far-field partner. Because partial hypotheses are rendered on persistent sides of a dialogue UI, speaker roles cannot be permuted and each speaker tag must precede its turn. These sparse tags are weakly optimized by supervised fine-tuning (SFT), while committing to a leading tag can delay the text that follows. We therefore post-train a streaming LLM-based ASR model with reinforcement learning with verifiable rewards (RLVR), combining a fixed-role Speaker-Attributed WER (SA-WER) grader with a turn-level onset-delay reward. On an eight-direction multi-talker cohort, SA-WER RL reduces SA-WER from $20.51\%$ for SFT to $13.14\%$. Adding the delay objective reduces onset $p50$ by $65\%$ (from $0.918\,\text{s}$ to $0.320\,\text{s}$) and $p90$ by $44\%$ (from $1.389\,\text{s}$ to