首页 > AI前沿 > Cord: Confidence-Anchored Order Residual Decoding for Masked Diffusion Language Models

Cord: Confidence-Anchored Order Residual Decoding for Masked Diffusion Language Models

arXiv自然语言 2026-10-29 13:03 3 阅读 查看原文
用户现在需要我把这段英文新闻正文排版成符合要求的HTML片段。首先,先看内容有没有自然的分段或小节。原文是一段连续的,但可以按内容逻辑拆分吗?不对,原文是一个长段落,但根据要求,长文用h2/h3划分,但这里原文没有明显的小节标题。不过再看,原文的内容结构:先讲问题,然后提出方法Cord,再讲方法特点、实验结果、行为审计、参数和延迟。不过原文是连贯的一段,所以可能需要拆成几个p?不对,原文每个自然段落用p包裹,原文只有一个自然段落吗?不对,仔细看原文: 第一句讲dLLM的解码顺序影响质量,但常见解码器用相同规则。 第二句讲置信度是强默认,但衡量的是当下预测的果断性,不是揭示该位置对后续解码的有用性。 第三句讲提出Cord,轻量提示条件策略,保留置信度解码作为稳定参考,学习何时何地通过生成方向的几何残差、接近已揭示上下文、双边填充来偏离。 第四句讲残差适应提示和演变的部分响应。 第五句讲Cord不是每个解码步骤采样新策略动作,而是每个rollout采样单个低维系数矩阵,诱导整个状态依赖的揭示轨迹。 第六句讲这带来上下文赌博机公式,直接从序列级奖励训练,无需步骤级信用分配或主干更新。 第七句讲在三个dLLM和五个推理、代码、约束满足基准上,Cord将平均任务成功率从32.0%提升到48.2%,进一步超过最佳每域固定残差。 第八句讲行为审计显示学习的残差诱导揭示局部性和双边填充的显著、任务依赖变化。 第九句讲Cord仅增加0.53M可训练参数,无需额外主干前向传播,仅带来5.39%推理延迟开销。 现在按自然段落,原文是一个自然段吗?但根据要求,长文可以用h2/h3划分小节。那可以给几个小节加标题,比如:

背景与动机

The decoding order...how useful revealing that position will be for subsequent decoding.

方法:Cord策略

We propose Cord...evolving partial response.

Rather than sampling...backbone updates.

实验结果

Across three dLLMs...best per-domain fixed residual.

行为审计与效率

Behavioral audits show...inference latency overhead.

但等一下,要求是“保留原文所有信息,不得增删改内容”,加h2的标题算不算增?不对,规则2说“长文用

/

划分小节”,但原文没有小节标题,那是不是可以不加?或者因为原文是长段落,需要拆分?不对,再看规则2:“每个自然段落用

包裹,长文用

/

划分小节”,也就是说,如果原文有自然的长文,用h2/h3,但原文没有小节,所以可能直接拆成几个p?不对,原文是一个连贯的段落,但可以按内容拆分p吗?比如:

The decoding order of masked diffusion language models (dLLMs) can substantially affect generation quality, yet common decoders typically apply the same reveal rule throughout generation. Confidence provides a strong default, but it measures how decisively a position can be predicted now, not how useful revealing that position will be for subsequent decoding.

We propose Cord, a lightweight prompt-conditioned policy that retains confidence decoding as a stable reference and learns when and how to depart from it through geometric residuals over generation direction, proximity to revealed context, and two-sided infilling. The residuals adapt to both the prompt and the evolving partial response.

Rather than sampling a new policy action at every decoding step, Cord samples a single low-dimensional coefficient matrix per rollout, which induces an entire state-dependent reveal trajectory. This yields a contextual-bandit formulation trained directly from sequence-level rewards, without step-level credit assignment or backbone updates.

Across three dLLMs and five reasoning, code, and constraint-satisfaction benchmarks, Cord improves average task success from 32.0% under confidence decoding to 48.2%, and further improves over the best per-domain fixed residual. Behavioral audits show that the learned residuals induce substantial, task-dependent changes in reveal locality and two-sided infilling.

Cord adds only $0.53$M trainable parameters, requires no additional backbone forward passes, and incurs only 5.39% inference latency overhead.

这样拆分是按内容逻辑,每个p是一个小的主题,符合要求。然后检查有没有需要用其他标签的地方:比如重要句子用strong,比如“substantially affect generation quality”“Confidence provides a strong default”“Cord”第一次出现可以加strong吗?规则3说重要句子可用strong,所以比如第一次出现Cord可以加Cord,还有“retains confidence decoding as a stable reference”“samples a single low-dimensional coefficient matrix per rollout”“contextual-bandit formulation”“sequence-level rewards”“improves average task success from 32.0% under confidence decoding to 48.2%”“substantial, task-dependent changes in reveal locality and two-sided infilling”“only $0.53$M trainable parameters”“no additional backbone forward passes”“only 5.39% inference latency overhead”这些重要的点可以用strong。比如: 第一个p里:The decoding order of masked diffusion language models (dLLMs) can substantially affect generation quality, yet common decoders typically apply the same reveal rule throughout generation. Confidence provides a strong default, but it measures how decisively a position can be predicted now, not how useful revealing that position will be for subsequent decoding. 第二个p里:We propose Cord, a lightweight prompt-conditioned policy that retains confidence decoding as a stable reference and learns when and how to depart from it through geometric residuals over generation direction, proximity to revealed context, and two-sided infilling. The residuals adapt to both the prompt and the evolving partial response. 第三个p里:Rather than sampling a new policy action at every decoding step, Cord samples a single low-dimensional coefficient matrix per rollout, which induces an entire state-dependent reveal trajectory. This yields a contextual-bandit formulation trained directly from sequence-level rewards, without step-level credit assignment or backbone updates. 第四个p里:Across three dLLMs and five reasoning, code, and constraint-satisfaction benchmarks, Cord improves average task success from 32.0% under confidence decoding to 48.2%, and further improves over the best per-domain fixed residual. Behavioral audits show that the learned residuals induce substantial, task-dependent changes in reveal locality and two-sided infilling. 第五个p里:Cord adds only $0.53$M trainable parameters, requires no additional backbone forward passes, and incurs only 5.39% inference latency overhead. 这样符合规则3的重要句子用strong。然后检查有没有列表、引用、代码,这里没有,所以不用。最后组合起来就是:

The decoding order of masked diffusion language models (dLLMs) can substantially affect generation quality, yet common decoders typically apply the same reveal rule throughout generation. Confidence provides a strong default, but it measures how decisively a position can be predicted now, not how useful revealing that position will be for subsequent decoding.

We propose Cord, a lightweight prompt-conditioned policy that retains confidence decoding as a stable reference and learns when and how to depart from it through geometric residuals over generation direction, proximity to revealed context, and two-sided infilling. The residuals adapt to both the prompt and the evolving partial response.

Rather than sampling a new policy action at every decoding step, Cord samples a single low-dimensional coefficient matrix per rollout, which induces an entire state-dependent reveal trajectory. This yields a