首页 > AI前沿 > Can Decision Models Replace LLMs in Real-World Contact Center Classification and Judging Tasks?

Can Decision Models Replace LLMs in Real-World Contact Center Classification and Judging Tasks?

arXiv自然语言 2026-11-17 23:39 8 阅读 查看原文
用户需要我作为一个专业的新闻排版工程师,将提供的一段纯文本新闻正文排版成语义化的HTML片段。 规则如下: 1. 只输出HTML片段本身,不要Markdown围栏,不要解释。 2. 每个自然段落用`

`包裹,长文用`

`/`

`划分小节。 3. 列表内容用`
  • `,引用用`
    `,代码用`
    `,重要句子可用``。
    4. 保留原文所有信息,不得增删改内容。
    5. 不要输出``/``/``等文档级标签。
    
    分析正文内容:
    正文是一段关于联系中心系统决策模型与生成式大语言模型比较的学术/技术新闻摘要。
    内容可以分为几个逻辑部分:
    1. 介绍联系中心系统需要做有界决策,决策模型的优势,但在实际应用中的效果尚不明确。
    2. 论文的实验设计:比较9个决策模型和13个生成式LLM,涵盖8个用例(6个分类,2个判断)。
    3. 实验结果:决策模型在分类任务上可匹配前沿LLM,API闭源决策模型成本最低;自信预测更准,支持选择性路由。
    4. 决策模型的弱点:分类任务中的弃权问题(很少选择无答案,除了Gemini-Decision-1);判断任务中较小模型质量下降且存在位置偏差。
    5. 总结:为基于弃权行为、置信度和部署成本选择决策模型提供实践指导,而不仅是整体准确率。
    
    由于这是一段不长的文本,我可以将其划分为几个小节,或者就作为一个段落但加上一些重点。为了符合“长文用`

    `/`

    `划分小节”的规则,我可以把正文拆分成几个逻辑小节。 拆分方案:

    研究背景与实验设计

    Contact center systems often need to make bounded decisions, such as identifying a caller's intent, retrieving the knowledge-base article that answers a query, assessing agent compliance, or judging the output of AI models used in production. Decision models return a typed answer and its probability directly, making them a promising cost-effective alternative to generative Large Language Models (LLMs) in such bounded conversational tasks. However, their effectiveness in real-world contact center applications remains unclear. In this paper, we compare nine decision models against thirteen generative LLMs in zero-shot settings across eight contact center use cases: six classification tasks and two judging tasks.

    实验结果与优势

    Our experiments demonstrate that many decision models can match the performance of frontier LLMs on contact center classification, and API-based closed decision models achieve this at the lowest per-request cost among the API models we evaluate. We further observe that their confident predictions are more accurate, enabling selective routing in real-world production scenarios.

    局限性与挑战

    Across classification tasks, the main weakness of most decision models is abstention: they rarely choose the no-answer option, which accounts for most of their gap to the strongest LLMs (although Gemini-Decision-1 is the notable exception in this regard). We also find that the judging quality of decision models drops sharply for smaller models, and they are also more prone to position bias.

    研究结论

    In short, our findings provide practical guidance for selecting decision models based on abstention behavior, confidence, and deployment cost, rather than overall accuracy alone.

    检查规则: 1. 只输出HTML片段,无Markdown,无解释。 -> 满足 2. 每个自然段落用`

    `包裹,长文用`

    `/`

    `划分小节。 -> 满足,使用了`

    `和`

    `。 3. 列表、引用、代码等(此处没有列表、引用、代码,但使用了``标记重要句子)。 -> 满足 4. 保留原文所有信息,不得增删改内容。 -> 检查文本是否完全一致。 原文: Contact center systems often need to make bounded decisions, such as identifying a caller's intent, retrieving the knowledge-base article that answers a query, assessing agent compliance, or judging the output of AI models used in production. Decision models return a typed answer and its probability directly, making them a promising cost-effective alternative to generative Large Language Models (LLMs) in such bounded conversational tasks. However, their effectiveness in real-world contact center applications remains unclear. In this paper, we compare nine decision models against thirteen generative LLMs in zero-shot settings across eight contact center use cases: six classification tasks and two judging tasks. Our experiments demonstrate that many decision models can match the performance of frontier LLMs on contact center classification, and API-based closed decision models achieve this at the lowest per-request cost among the API models we evaluate. We further observe that their confident predictions are more accurate, enabling selective routing in real-world production scenarios. Across classification tasks, the main weakness of most decision models is abstention: they rarely choose the no-answer option, which accounts for most of their gap to the strongest LLMs (although Gemini-Decision-1 is the notable exception in this regard). We also find that the judging quality of decision models drops sharply for smaller models, and they are also more prone to position bias. In short, our findings provide practical guidance for selecting decision models based on abstention behavior, confidence, and deployment cost, rather than overall accuracy alone. 对比我的HTML内容: 段落1完全一致。 段落2:Our experiments demonstrate that many decision models can match the performance of frontier LLMs on contact center classification, and API-based... 一致。 段落3完全一致。 段落4完全一致。 确保没有增删改任何文字,除了添加HTML标签和``。 注意:原文是4个自然段。我加了4个`

    `标题,这算不算“增删改内容”?规则说“保留原文所有信息,不得增删改内容”,同时规则2说“长文用`

    `/`

    `划分小节”。所以加标题是为了排版,不视为改变正文内容。为了更稳妥,我可以直接用`

    `或者不硬加标题,但规则明确要求“长文用`

    `/`

    `划分小节”,所以加小节标题是符合排版要求的。或者我可以不添加额外的标题文字,而是直接用标签?不,小节标题需要文字。我可以使用概括性的文字作为小标题,这属于排版工程的常规操作,不违反“保留原文信息”的核心。 或者,如果不加标题,直接把四个自然段用`

    `包裹,也是符合“每个自然段落用`

    `包裹”的。但为了体现“长文用`

    `/`

    `划分小节”,我还是加上合理的小标题。 小标题:

    背景与实验设计

    实验表现与优势

    模型局限性

    研究结论

    这样既满足了排版规则,又没有篡改正文内容。 再检查一遍HTML:

    研究背景与实验设计

    Contact center systems often need to make bounded decisions, such as identifying a caller's intent, retrieving the knowledge-base article that answers a query, assessing agent compliance, or judging the output of AI models used in production. Decision models return a typed answer and its probability directly, making them a promising cost-effective alternative to generative Large Language Models (LLMs) in such bounded conversational tasks. However, their effectiveness in real-world contact center applications remains unclear. In this paper, we compare nine decision models against thirteen generative LLMs in zero-shot settings across eight contact center use cases: six classification tasks and two judging tasks.

    实验结果与核心优势

    Our experiments demonstrate that many decision models can match the performance of frontier LLMs on contact center classification, and API-based closed decision models achieve this at the lowest per-request cost among the API models we evaluate. We further observe that their confident predictions are more accurate, enabling selective routing in real-world production scenarios.

    存在的局限与挑战

    Across classification tasks, the main weakness of most decision models is abstention: they rarely choose the no-answer option, which accounts for most of their gap to the strongest LLMs (although Gemini-Decision-1 is the notable exception in this