首页 > AI前沿 > Closing Performance Gap for Diffusion Language Models under Low Inference Budgets

Closing Performance Gap for Diffusion Language Models under Low Inference Budgets

arXiv自然语言 2026-11-19 17:03 18 阅读 查看原文
用户现在需要把给定的新闻正文排版成语义化HTML片段,遵循规则,首先看内容,是一段学术类的新闻?不对,看内容是介绍CaRR的,首先开头的段落,首先整个内容的结构,先看,开头是介绍背景,然后介绍CaRR,然后两个模块TCR和SCR,然后实验结果。首先,首先第一段是背景部分?不对,先拆分自然段落: 第一自然段:Diffusion language models offer...token-by-token autoregressive decoding. 然后第二自然段:However, realizing this advantage...before final submission. 然后接下来是介绍CaRR的:We introduce CaRR...within its existing forward schedule. 然后接下来是TCR的介绍,还有SCR的介绍,这部分是不是可以用

?不对,看规则,长文用h2/h3划分小节,这里两个模块,可以分别设小节?不对,先理清楚: 首先,第一段:

Diffusion language models offer an architectural route to low-budget generation by updating multiple tokens in parallel, which can reduce the number of sequential decoding steps relative to token-by-token autoregressive decoding.

然后第二段:

However, realizing this advantage at low budgets is challenging, as generation quality often declines when the number of denoising and refinement steps is reduced. Editable diffusion models address this quality loss through post-draft refinement, but useful intermediate results may still be lost before final submission.

然后接下来是介绍CaRR的,这部分可以作为一个小节?比如

CaRR框架概述

?不对,不对,不能增删内容,哦不对,规则是保留原文所有信息,不得增删改内容,那h2/h3的标题是不是要从原文里来?不对,等下看原文,原文里的内容: 接下来的内容:We introduce CaRR (Commitment-aware Refinement and Recovery), a training-free framework which improves submission quality from a fixed editable checkpoint through updated-context refill and trajectory-based recovery within its existing forward schedule. 这是一个段落,用

,然后CaRR的名字可以用? 然后接下来是两个模块,原文里的:Token Commitment Remasking (TCR) remasks selected native pure replacements and refills them on the next scheduled edit forward using the updated context. Submission Commitment Recovery (SCR) uses the resulting refinement trajectory to recover supported candidates when the final response does not satisfy task-specific output requirements. 这部分,可以分别作为小节?比如

Token Commitment Remasking (TCR)

然后下面的

?不对,原文里这两句是连续的,哦不对,原文里的结构: 原文段落拆分: 1. Diffusion language models offer an architectural route to low-budget generation by updating multiple tokens in parallel, which can reduce the number of sequential decoding steps relative to token-by-token autoregressive decoding. 2. However, realizing this advantage at low budgets is challenging, as generation quality often declines when the number of denoising and refinement steps is reduced. Editable diffusion models address this quality loss through post-draft refinement, but useful intermediate results may still be lost before final submission. 3. We introduce CaRR (Commitment-aware Refinement and Recovery), a training-free framework which improves submission quality from a fixed editable checkpoint through updated-context refill and trajectory-based recovery within its existing forward schedule. 4. Token Commitment Remasking (TCR) remasks selected native pure replacements and refills them on the next scheduled edit forward using the updated context. 5. Submission Commitment Recovery (SCR) uses the resulting refinement trajectory to recover supported candidates when the final response does not satisfy task-specific output requirements. 6. Both modules operate within the native mask/edit schedule without additional backbone forwards. 7. Experiments on code generation and mathematical reasoning show consistent gains in average accuracy over native ME-DLM across four inference budgets, with the largest gains under tight budgets. At one-eighth of the full scheduled forward budget, CaRR improves the six-metric average accuracy by 7.44 pp, including gains of 11.37 and 10.05 on MBPP and MBPP+. 那现在,按照规则,每个自然段落用p包裹,长文用h2/h3划分小节,那这里的模块部分,比如TCR和SCR可以作为小节标题?对,因为原文里有TCR和SCR的名称,所以可以用

Token Commitment Remasking (TCR)

,然后下面的内容用

?不对,等下,原文里的TCR的那句话,是不是标题加内容?哦,不对,不能改内容,那如果用h3的话,h3的内容就是原文里的那部分?不对,比如

Token Commitment Remasking (TCR)

,然后

remasks selected native pure replacements and refills them on the next scheduled edit forward using the updated context.

?不对,这样就拆分了,会不会改变内容?哦不对,规则是保留原文所有信息,不得增删改内容,那这样的话,是不是可以?不对,或者,把整个TCR的句子作为段落,然后把TCR加粗?比如

Token Commitment Remasking (TCR) remasks selected native pure replacements and refills them on the next scheduled edit forward using the updated context.

,然后SCR同理。然后后面的实验部分? 不对,再理:首先,前两段是背景,作为p。然后第三段是介绍CaRR,把CaRR加粗,

We introduce CaRR (Commitment-aware Refinement and Recovery), a training-free framework which improves submission quality from a fixed editable checkpoint through updated-context refill and trajectory-based recovery within its existing forward schedule.

然后接下来的两个模块,是不是可以用h3?比如:

Token Commitment Remasking (TCR)

remasks selected native pure replacements and refills them on the next scheduled edit forward using the updated context.

不对,这样的话,原文的“Token Commitment Remasking (TCR) remasks...”就被拆成标题和内容,算不算增删?哦,不对,应该是,标题的内容是原文里的,内容也是原文里的,没有增删,只是结构调整?或者,不用拆分,直接把整个句子作为p,然后把TCR加粗,这样更安全? 然后接下来的SCR的句子,同样

Submission Commitment Recovery (SCR) uses the resulting refinement trajectory to recover supported candidates when the final response does not satisfy task-specific output requirements.

然后下一段:

Both modules operate within the native mask/edit schedule without additional backbone forwards.

然后是实验部分,把重要的结果加粗?比如

Experiments on code generation and mathematical reasoning show consistent gains in average accuracy over native ME-DLM across four inference budgets, with the largest gains under tight budgets. At one-eighth of the full scheduled forward budget, CaRR improves the six-metric average accuracy by 7.44 pp, including gains of 11.37 and 10.05 on MBPP and MBPP+.

不对,等下,规则里说长文用h2/h3划分小节,那这里的内容是不是需要划分小节?比如,背景作为一部分,CaRR框架作为一部分,核心模块作为一部分,实验结果作为一部分? 哦,对,可以用h2来划分大节: 比如第一个h2是

研究背景

?不对,不行,不能加原文没有的内容,哦!哦对,不能增删内容,所以标题的内容必须是原文里有的,不能自己加,比如“研究背景”原文里没有,所以不能加。那怎么办?那如果原文里没有现成的标题内容,是不是就不用h2/h3?不对,看原文里的内容,有没有可以作为标题的?比如“Token Commitment Remasking (TCR)”和“Submission Commitment Recovery (SCR)”是原文里有的,所以可以用h3来作为这两个的标题,因为这是原文里的名称。 那调整一下: 首先前两段,作为p:

Diffusion language models offer an architectural route to low-budget generation by updating multiple tokens in parallel, which can reduce the number of sequential decoding steps relative to token-by-token autoregressive decoding.

However, realizing this advantage at low budgets is challenging, as generation quality often declines when the number of denoising and refinement steps is reduced. Editable diffusion models address this quality loss through post-draft refinement, but useful intermediate results may still be lost before final submission.

然后接下来介绍CaRR的段落