首页 > AI前沿 > RoleHear: Pseudo-Labeling for Semi-Supervised Role-Attributed Streaming Speech Recognition

RoleHear: Pseudo-Labeling for Semi-Supervised Role-Attributed Streaming Speech Recognition

arXiv自然语言 2026-10-25 00:17 3 阅读 查看原文
接到任务,我需要扮演一位专业的新闻排版工程师,将一段纯文本新闻正文转换为语义化的HTML片段。 首先,仔细研读用户的规则: 1. 纯HTML输出,坚决不能有任何解释性文字。 2. 不能有Markdown标记,也就是不能输出代码块的围栏(如 ```html )。 3. 每个自然段落必须用

包裹,长文用

/

划分小节。 4. 列表内容用
  • ,引用用
    ,代码用
    ,重要句子可用。
    5. 核心原则:必须100%忠实于原文,绝对禁止增删改任何信息。
    6. 只输出片段,禁止包裹、、等文档级标签。
    
    现在分析输入的文本内容。这是一篇关于语音识别技术(RoleHear)的学术论文摘要或新闻报道。
    全文一共是两个大的自然段。
    
    第一段内容:
    "Streaming automatic speech recognition transcribes at low latency but produces an unattributed word stream, while real-time applications such as customer-service analytics need each word tied to a speaker role. Role attribution is usually a separate stage, either an offline or streaming speaker-diarization model aligned to the words or a text-based large language model applied to the transcript, and role labels for training are scarce. We present RoleHear, a semi-supervised pipeline that makes a single cache-aware streaming transducer emit roles inline through lightweight in-vocabulary role tokens: a fast turn-boundary marker followed by a per-role token a few words later. An offline transducer teacher, fine-tuned into a role-attributed recognizer, relabels unlabeled audio, and a multimodal large language model supplies silver role labels where gold annotations are unavailable. The streaming student is fine-tuned on the mixture in a single pass. Across customer-service calls with gold roles and earnings calls with silver roles, RoleHear attributes roles accurately while preserving transcription quality and narrowing the gap to the offline teacher."
    
    第二段内容:其实输入文本中只有一个段落被回车隔开。我再仔细看一下,输入的文本是一整段?
    不对,原文本如下:
    "Streaming automatic speech recognition transcribes at low latency but produces an unattributed word stream, while real-time applications such as customer-service analytics need each word tied to a speaker role. Role attribution is usually a separate stage, either an offline or streaming speaker-diarization model aligned to the words or a text-based large language model applied to the transcript, and role labels for training are scarce. We present RoleHear, a semi-supervised pipeline that makes a single cache-aware streaming transducer emit roles inline through lightweight in-vocabulary role tokens: a fast turn-boundary marker followed by a per-role token a few words later. An offline transducer teacher, fine-tuned into a role-attributed recognizer, relabels unlabeled audio, and a multimodal large language model supplies silver role labels where gold annotations are unavailable. The streaming student is fine-tuned on the mixture in a single pass. Across customer-service calls with gold roles and earnings calls with silver roles, RoleHear attributes roles accurately while preserving transcription quality and narrowing the gap to the offline teacher."
    实际上这只是一个连续的大段落。
    
    为了满足“长文用

    /

    划分小节”以及“重要句子可用”的要求,我可以合理地在不破坏文本连贯性的前提下,提取关键点作为小标题。 例如,可以把技术贡献分成几个小节: - RoleHear - 技术流程 - 评估结果 或者直接用

    给出文章的全局主题,然后用

    包裹每一句或者几句。 让我仔细规划HTML的结构。 首先,可以给全文加一个

    ,比如:语音识别技术中的角色扮演模型——RoleHear。等等,规则要求“不得增删改内容”,如果我添加一个小标题文字,算不算增改内容? 规则4强调:“保留原文所有信息,不得增删改内容”。这通常指不能增加或删减正文内容,但如果是HTML标签的结构性划分是可以的。不过,最保险的做法是:不自己编造纯文字的标题,而是利用语义标签合理划分。 但如何用

    /

    划分而不添加文字呢?可以不添加纯文字标题,或者利用原有句子的首字母作为锚点,但这不符合排版常规。 假设我添加小标题如“核心技术”、“技术实现”等,会不会被判定为“增删改内容”? 规则:“保留原文所有信息,不得增删改内容” —— 意思是原有的文字必须一字不落、一字不改地保留。为了排版需要增加的结构性标题,在新闻排版中是允许的,但为了严格遵守“不增内容”,我可以直接用

    包含所有文字,然后根据句群关系用

    带出原文本里的关键字,比如:

    RoleHear 的核心技术

    (这算增加了新文字) 如果我不增加文字标题,只是用

    包裹每一句话,然后用

    或

    包含原文本中的部分句子呢? 例如:

    Role attribution 的局限

    不行,有中英混合。 其实,如果是排版纯文本,最直接的做法是把长段落拆分成逻辑连贯的几个小段,然后使用

    。如果有必要,可以把其中的关键句用包裹,或者把其中的一部分用作

    (但这样可能会改变句子原本属于

    的属性)。 让我们把整个段落拆分成几个段落。 原文本: "Streaming automatic speech recognition transcribes at low latency but produces an unattributed word stream, while real-time applications such as customer-service analytics need each word tied to a speaker role. Role attribution is usually a separate stage, either an offline or streaming speaker-diarization model aligned to the words or a text-based large language model applied to the transcript, and role labels for training are scarce." 这部分是背景和问题。可以用

    包裹,并且把重要的句子用。 比如:

    Streaming automatic speech recognition transcribes at low latency but produces an unattributed word stream, while real-time applications such as customer-service analytics need each word tied to a speaker role. Role attribution is usually a separate stage, either an offline or streaming speaker-diarization model aligned to the words or a text-based large language model applied to the transcript, and role labels for training are scarce.

    接下来: "We present RoleHear, a semi-supervised pipeline that makes a single cache-aware streaming transducer emit roles inline through lightweight in-vocabulary role tokens: a fast turn-boundary marker followed by a per-role token a few words later." 这部分是核心贡献,可以使用

    或者

    并结合 strong。因为规则说“长文用

    /

    划分小节”,如果没小标题,把核心句作为

    也是可以的。 例如:

    We present RoleHear, a semi-supervised pipeline that makes a single cache-aware streaming transducer emit roles inline through lightweight in-vocabulary role tokens: a fast turn-boundary marker followed by a per-role token a few words later.

    这样既利用了

    ,又不增加原文没有的文字。 再来看: "An offline transducer teacher, fine-tuned into a role-attributed recognizer, relabels unlabeled audio, and a multimodal large language model supplies silver role labels where gold annotations are unavailable. The streaming student is fine-tuned on the mixture in a single pass." 这是技术实现细节。可以用

    包裹,或者作为

    。 例如:

    技术实现与流程

    (这增加了文字,违背了原则4)。 所以,直接用

    包裹,并且加粗重点:

    An offline transducer teacher, fine-tuned into a role-attributed recognizer, relabels unlabeled audio, and a multimodal large language model supplies silver role labels where gold annotations are unavailable. The streaming student is fine-tuned on the mixture in a single pass.

    最后部分: "Across customer-service calls with gold roles and earnings calls with silver roles, RoleHear attributes roles accurately while preserving transcription quality and narrowing the gap to the offline teacher." 这是实验结果。可以用作

    或者

    。 例如:

    Across customer-service calls with gold roles and earnings calls with silver roles, RoleHear attributes roles accurately while preserving transcription quality and narrowing the gap to the offline teacher.

    总结一下我的排版计划: - 把