首页 > AI前沿 > One Model, Two Paths: Balancing Single-Pass and Looped Performance in Language Models

One Model, Two Paths: Balancing Single-Pass and Looped Performance in Language Models

arXiv自然语言 2026-10-24 15:50 3 阅读 查看原文
The user wants me to format a plain text news article into a semantic HTML fragment. Rules: 1. Only output the HTML fragment itself, no Markdown fences, no explanations. 2. Each natural paragraph wrapped in `

`, long text divided into sections with `

`/`

`. 3. Lists use `
  • `, quotes use `
    `, code uses `
    `, important sentences can use ``.
    4. Retain all original information, no adding, deleting, or modifying content.
    5. No document-level tags like ``, ``, ``.
    
    Let's analyze the source text:
    "Repeating Transformer layers can improve language-model performance without adding parameters. A pretrained model can be adapted to support both single-pass and looped inference by reusing middle-layer weights for additional computation. Because both paths share weights, training to improve looped predictions can degrade single-pass quality. We study joint training strategies using weighted losses on both outputs, with guidance from the frozen original model. Joint supervision substantially reduces single-pass degradation compared to looped-only training, while preserving looped gains over ordinary continued training. Most of this benefit comes from adjusting the single-pass loss weight, with guidance from the frozen original model further improving the looped output. With validation-selected weights, our method preserves single-pass quality within a predefined tolerance of ordinary continued training while improving looped predictions."
    
    This text is essentially one long paragraph, but I can break it into logical sections or paragraphs to make it semantic.
    Wait, the rule says "每个自然段落用

    包裹,长文用

    /

    划分小节". The text is short (about 7 sentences). I can just wrap it in `

    ` tags, or add some headings if it makes sense, but since it's a short abstract, maybe just `

    ` is enough, or I can add `

    ` for title/sections if I split it. Wait, rule 4 says "保留原文所有信息,不得增删改内容". If I add headings, is that adding content? Usually, adding structural headings is okay if it's just for semantic structure, but to be safe, I can just use `

    ` for the paragraphs. Let's split the text into a few paragraphs based on logical breaks, or just keep it as one `

    ` and use `` for important sentences. Wait, "每个自然段落用

    包裹" (Wrap each natural paragraph in

    ). The original text is one single paragraph. I will wrap it in one `

    `. Let's identify important sentences to wrap in ``. - "Repeating Transformer layers can improve language-model performance without adding parameters." -> not necessarily. - "Because both paths share weights, training to improve looped predictions can degrade single-pass quality." -> important. - "We study joint training strategies using weighted losses on both outputs, with guidance from the frozen original model." -> important. - "Joint supervision substantially reduces single-pass degradation compared to looped-only training, while preserving looped gains over ordinary continued training." -> very important. - "With validation-selected weights, our method preserves single-pass quality within a predefined tolerance of ordinary continued training while improving looped predictions." -> very important. Let's just use `

    ` and ``. Text:

    Repeating Transformer layers can improve language-model performance without adding parameters. A pretrained model can be adapted to support both single-pass and looped inference by reusing middle-layer weights for additional computation. Because both paths share weights, training to improve looped predictions can degrade single-pass quality. We study joint training strategies using weighted losses on both outputs, with guidance from the frozen original model. Joint supervision substantially reduces single-pass degradation compared to looped-only training, while preserving looped gains over ordinary continued training. Most of this benefit comes from adjusting the single-pass loss weight, with guidance from the frozen original model further improving the looped output. With validation-selected weights, our method preserves single-pass quality within a predefined tolerance of ordinary continued training while improving looped predictions.

    Wait, is there any list, quote, or code? No. So just `

    ` and ``. Let's review the exact text to ensure no changes: "Repeating Transformer layers can improve language-model performance without adding parameters. A pretrained model can be adapted to support both single-pass and looped inference by reusing middle-layer weights for additional computation. Because both paths share weights, training to improve looped predictions can degrade single-pass quality. We study joint training strategies using weighted losses on both outputs, with guidance from the frozen original model. Joint supervision substantially reduces single-pass degradation compared to looped-only training, while preserving looped gains over ordinary continued training. Most of this benefit comes from adjusting the single-pass loss weight, with guidance from the frozen original model further improving the looped output. With validation-selected weights, our method preserves single-pass quality within a predefined tolerance of ordinary continued training while improving looped predictions." Let's check the words: Repeating Transformer layers can improve language-model performance without adding parameters. A pretrained model can be adapted to support both single-pass and looped inference by reusing middle-layer weights for additional computation. Because both paths share weights, training to improve looped predictions can degrade single-pass quality. We study joint training strategies using weighted losses on both outputs, with guidance from the frozen original model. Joint supervision substantially reduces single-pass degradation compared to looped-only training, while preserving looped gains over ordinary continued training. Most of this benefit comes from adjusting the single-pass loss weight, with guidance from the frozen original model further improving the looped output. With validation-selected weights, our method preserves single-pass quality within a predefined tolerance of ordinary continued training while improving looped predictions. I will format it as a single paragraph or split it into a few `

    ` tags if it looks better, but the original is one paragraph. "每个自然段落用

    包裹" means each natural paragraph. I'll just use one `

    `. Wait, to make it look more like a "news" layout, maybe I should add an `

    ` title? But rule 4 says "不得增删改内容" (must not add, delete, or modify content). Adding a title would be adding content. So I'll stick to just the text provided. Let's use `` for the key findings:

    Repeating Transformer layers can improve language-model performance without adding parameters. A pretrained model can be adapted to support both single-pass and looped inference by reusing middle-layer weights for additional computation. Because both paths share weights, training to improve looped predictions can degrade single-pass quality. We study joint training strategies using weighted losses on both outputs, with guidance from the frozen original model. Joint supervision substantially reduces single-pass degradation compared to looped-only training, while preserving looped gains over ordinary continued training. Most of this benefit comes from adjusting the single-pass loss weight, with guidance from the frozen original model further improving the looped output. With validation-selected weights, our method preserves single-pass quality within a predefined tolerance of ordinary continued training while improving looped predictions.

    Check if there are any subtle formatting rules I missed. "长文用

    /

    划分小节" - it's a short text, so no need for headings. "列表内容用
    • ,引用用
      ,代码用
      " - none present.
      "重要句子可用" - done.
      "不要输出//等文档级标签" - done.
      "只输出HTML片段本身,不要Markdown围栏、不要解释" - done.
      
      Looks good.
      Double check the exact wording:
      "Repeating Transformer layers can improve language-model performance without adding parameters. A pretrained model can be adapted to support both single-pass and looped inference by reusing middle-layer weights for additional computation. Because both paths share weights, training to improve looped predictions can degrade single-pass quality. We study joint training strategies using weighted losses on both outputs, with guidance from the frozen original model. Joint supervision substantially reduces single-pass degradation compared to looped-only training, while preserving looped gains over ordinary continued training. Most of this benefit comes from adjusting the