首页 > AI前沿 > BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion

BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion

arXiv自然语言 2026-10-30 19:04 3 阅读 查看原文
用户现在让我把给定的新闻正文排版成语义化的HTML片段。首先,先看正文是一个长段落,需要用

划分小节吗?先看内容结构:首先讲DLM和AR模型的不同,然后提出BYOD方法,接着讲转换的模型,参数占比,资源需求,效果评估,限制,最后提供资源和结论。首先,标题部分?原文开头是介绍背景,然后方法,所以可能用

把关键部分分一下?不对,先看用户要求,长文用

/

划分小节。首先,原文可以分成几个部分:1. 背景与方法概述;2. 转换模型与资源消耗;3. 效果评估与限制;4. 资源提供与结论。然后每个部分用

包裹?或者直接按自然段落?不对,原文是一个长段落,但里面有不同的内容块。首先,第一句讲AR和DLM的不同,然后讲BYOD方法,然后讲转换的模型,然后讲参数和训练资源,然后讲生成效果,然后讲限制,然后讲提供的资源,最后结论。那首先,用

来划分小节,比如:

Background and Proposed Method

然后对应的

,然后

Conversion Results and Resource Consumption

,然后

,然后

Generation Performance and Limitations

,然后

,然后

Released Resources and Conclusion

,然后

?等一下,要保留原文所有信息,不得增删改。那原文的结构:首先,第一部分:“Unlike autoregressive (AR) models, diffusion language models (DLMs) permit parallel token updates, flexible generation, and bidirectional attention, but leading systems rely on costly pretraining or full-model fine-tuning. We present Build Your Own DLM (BYOD), a generalizable LoRA-only recipe for parameter-efficient AR-to-DLM conversion.” 这部分是背景和方法,所以可以用

Context and Method

,然后

包裹这部分?然后下一部分:“Without architecture-specific tuning, BYOD induces diffusion generation in Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B; to our knowledge, this is the first reported AR-to-DLM conversion of a Mistral model. The adapters comprise 4.1-5.8% of final parameters while every base-model parameter remains frozen; each 25,000-iteration conversion needs at most 102.4M tokens, processed in 3.1-4.0 hours on one Google Colab GPU.” 这部分是转换的模型和资源,所以

Conversion Targets and Resource Usage

,然后

。然后下一部分:“All four models generate from fully masked sequences, use bidirectional context in a controlled attention test, and retain substantial parent-model knowledge. BYOD-Gemma is the strongest BYOD conversion on our nine-task evaluation, reaching 50.6% average accuracy compared with 51.9% for LLaDA 8B Instruct in the same harness.” 这是生成效果,然后“Open-generation quality at low network function evaluation (NFE) budgets remains the principal limitation: BYOD perplexity is 8.3-11.1 versus 4.1 for LLaDA and 2.2-3.0 for the AR parents when generating 128 tokens in 64 steps.” 这是限制,所以可以用

Performance Evaluation and Limitations

,然后

包含效果和限制?然后最后部分:“To enable further experimentation and inspection, we provide a standalone training and evaluation implementation and interactive demos for all four conversions. BYOD thus provides a practical route for extending diffusion generation to other existing model families while identifying the gap that efficient conversion must next close.” 这是资源和结论,用

Released Resources and Conclusion

,然后

。现在,检查每个部分:然后,重要的句子比如“BYOD induces diffusion generation in Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B; to our knowledge, this is the first reported AR-to-DLM conversion of a Mistral model.” 可以用吗?比如Without architecture-specific tuning, BYOD induces diffusion generation in Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B; to our knowledge, this is the first reported AR-to-DLM conversion of a Mistral model.?或者里面的“first reported AR-to-DLM conversion of a Mistral model”用?比如:“Without architecture-specific tuning, BYOD induces diffusion generation in Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B; to our knowledge, this is the first reported AR-to-DLM conversion of a Mistral model.” 然后,“BYOD-Gemma is the strongest BYOD conversion on our nine-task evaluation, reaching 50.6% average accuracy compared with 51.9% for LLaDA 8B Instruct in the same harness.” 可以用BYOD-Gemma is the strongest BYOD conversion吗?或者直接保留。现在,开始排版:首先,

Background and Method

Unlike autoregressive (AR) models, diffusion language models (DLMs) permit parallel token updates, flexible generation, and bidirectional attention, but leading systems rely on costly pretraining or full-model fine-tuning. We present Build Your Own DLM (BYOD), a generalizable LoRA-only recipe for parameter-efficient AR-to-DLM conversion.

Conversion Targets and Resource Consumption

Without architecture-specific tuning, BYOD induces diffusion generation in Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B; to our knowledge, this is the first reported AR-to-DLM conversion of a Mistral model. The adapters comprise 4.1-5.8% of final parameters while every base-model parameter remains frozen; each 25,000-iteration conversion needs at most 102.4M tokens, processed in 3.1-4.0 hours on one Google Colab GPU.

Generation Performance and Evaluation

All four models generate from fully masked sequences, use bidirectional context in a controlled attention test, and retain substantial parent-model knowledge. BYOD-Gemma is the strongest BYOD conversion on our nine-task evaluation, reaching 50.6% average accuracy compared with 51.9% for LLaDA 8B Instruct in the same harness.

Principal Limitation

Open-generation quality at low network function evaluation (NFE) budgets remains the principal limitation: BYOD perplexity is 8.3-11.1 versus 4.1 for LLaDA and 2.2-3.0 for the AR parents when generating 128 tokens in 64 steps.

Released Resources and Conclusion

To enable further experimentation and inspection, we provide a standalone training and evaluation implementation and interactive demos for all four conversions. BYOD thus provides a practical route for extending diffusion generation to other existing model families while identifying the gap that efficient conversion must next close.

这样对吗?检查规则:1. 只输出HTML片段,没有Markdown围栏,没有解释。2. 每个自然段落用

包裹,长文用

划分小节。3. 重要句子用,比如“the first reported AR-to-DLM conversion of a Mistral model”和“BYOD-Gemma is the strongest BYOD conversion”用。4. 保留原文所有信息,没有增删改。5. 没有文档级标签。再检查原文内容:所有句子都包含了吗?原文的每一句话都在对应的

里吗?比如:第一部分:“Unlike autoregressive (AR) models, diffusion language models (DLMs) permit parallel token updates, flexible generation, and bidirectional attention, but leading systems rely on costly pretraining or full-model fine-tuning. We present Build Your Own DLM (BYOD), a generalizable LoRA-only recipe for parameter-efficient AR-to-DLM conversion.” 正确,在第一个

。第二部分:“Without architecture-specific tuning, BYOD induces diffusion generation in