首页 > AI前沿 > P2N: Reusing Deep Representations for Greater Effective Depth

P2N: Reusing Deep Representations for Greater Effective Depth

arXiv自然语言 2026-10-20 12:00 3 阅读 查看原文
1. Analyze the Request: * Role: Professional news typesetting engineer. * Task: Convert plain text news body into semantic HTML. * Rules: 1. Output *only* the HTML fragment, no Markdown fences, no explanations. 2. Paragraphs use `

`, long articles use `

`/`

` for sections. 3. Lists use `
  • `, quotes use `
    `, code uses `
    `, important sentences use ``.
            4. Preserve *all* original information, no additions/de deletions.
            5. Do *not* output document-level tags like ``, ``, ``.
        *   Input Text: A short paragraph about "P2N" (Previous to Next) for increasing computational depth in LLMs.
    
    2.  Analyze the Input Text:
        "Mathematical problem solving and autonomous agents are increasingly important applications of language models, demanding stronger reasoning capabilities. Computational depth is a key resource for such reasoning, but adding model layers increases both training and inference costs, while test-time scaling incurs substantial additional inference computation. We introduce P2N (Previous to Next), which reuses the previous token's deep representations, already available during generation, to increase effective computational depth with negligible decoding overhead. Specifically, P2N adds the previous token's deep hidden state to the current token's shallow representation, allowing subsequent layers to build on previously completed deep computation. This recurrence extends effective computational depth across tokens, while introducing only a single vector addition per decoding step. We train the recurrence using parallel Jacobi updates restricted to a middle-layer core. Experiments with Qwen-style models show that, at 1.5B parameters, P2N improves GSM8K accuracy by over 10 percentage points under both zero-shot and few-shot prompting, with nearly unchanged decoding latency. The benefits also extend to language modeling and average performance on general downstream benchmarks. on general downstream benchmarks."
    
    3.  Drafting the HTML Structure:
        *   The text is a single block of continuous prose. It doesn't explicitly have section headers in the source text, but as a "long article" it could be broken down. However, rule 4 says "preserve all original information, no additions/deletions". Adding headers that aren't there might violate the spirit of preserving the exact text structure, but rule 2 says "长文用

    /

    划分小节" (long articles use h2/h3 to divide sections). Since this is a short abstract, I should probably just wrap it in `

    ` tags, or perhaps split it logically if needed. Let's look closely at the text. It's one cohesive paragraph describing the problem, the method, and the results. Splitting it into sections might require inventing headers, which violates rule 4 ("不得增删改内容"). So, keeping it as one or two `

    ` tags is safest, or maybe just one `

    ` since it's short. Wait, the prompt says "每个自然段落用

    包裹,长文用

    /

    划分小节". Since the input is just one paragraph, I will format it as a single `

    ` or maybe two if I can find a natural break. The text has a slight repetition at the end: "...and average performance on general downstream benchmarks. on general downstream benchmarks." This repetition is in the original text, so I must keep it. * Let's identify important sentences for ``: * "Computational depth is a key resource for such reasoning..." -> Maybe not strong enough. * "We introduce P2N (Previous to Next)..." -> Good candidate. * "Specifically, P2N adds the previous token's deep hidden state to the current token's shallow representation..." -> Good candidate. * "This recurrence extends effective computational depth across tokens, while introducing only a single vector addition per decoding step." -> Good candidate. * "Experiments with Qwen-style models show that, at 1.5B parameters, P2N improves GSM8K accuracy by over 10 percentage points under both zero-shot and few-shot prompting, with nearly unchanged decoding latency." -> Good candidate. * Let's check for lists, blockquotes, code: None present in the text. 4. Refining the HTML: Since there are no explicit lists, blockquotes, or code blocks, I will just use `

    ` tags and `` tags where appropriate. Wait, can I break the paragraph into multiple `

    ` tags? Yes, paragraphs are separated by newlines in markdown, but here it's a single block. I'll stick to one `

    ` or maybe two if it feels right, but strictly speaking, a single block of text is one paragraph. Let's make it one `

    `. Let's review the exact text to ensure no words are changed: "Mathematical problem solving and autonomous agents are increasingly important applications of language models, demanding stronger reasoning capabilities. Computational depth is a key resource for such reasoning, but adding model layers increases both training and inference costs, while test-time scaling incurs substantial additional inference computation. We introduce P2N (Previous to Next), which reuses the previous token's deep representations, already available during generation, to increase effective computational depth with negligible decoding overhead. Specifically, P2N adds the previous token's deep hidden state to the current token's shallow representation, allowing subsequent layers to build on previously completed deep computation. This recurrence extends effective computational depth across tokens, while introducing only a single vector addition per decoding step. We train the recurrence using parallel Jacobi updates restricted to a middle-layer core. Experiments with Qwen-style models show that, at 1.5B parameters, P2N improves GSM8K accuracy by over 10 percentage points under both zero-shot and few-shot prompting, with nearly unchanged decoding latency. The benefits also extend to language modeling and average performance on general downstream benchmarks. on general downstream benchmarks." HTML draft: ```html

    Mathematical problem solving and autonomous agents are increasingly important applications of language models, demanding stronger reasoning capabilities. Computational depth is a key resource for such reasoning, but adding model layers increases both training and inference costs, while test-time scaling incurs substantial additional inference computation. We introduce P2N (Previous to Next), which reuses the previous token's deep representations, already available during generation, to increase effective computational depth with negligible decoding overhead. Specifically, P2N adds the previous token's deep hidden state to the current token's shallow representation, allowing subsequent layers to build on previously completed deep computation. This recurrence extends effective computational depth across tokens, while introducing only a single vector addition per decoding step. We train the recurrence using parallel Jacobi updates restricted to a middle-layer core. Experiments with Qwen-style models show that, at 1.5B parameters, P2N improves GSM8K accuracy by over 10 percentage points under both zero-shot and few-shot prompting, with nearly unchanged decoding latency. The benefits also extend to language modeling and average performance on general downstream benchmarks. on general downstream benchmarks.

    ``` Wait, should I bold "stronger reasoning capabilities"? It's an adjective modifying capabilities, maybe just "stronger reasoning" or the whole sentence. The prompt says "重要句子可用" (important sentences can use ). I will bold key phrases or sentences that represent the core contributions/results. Let's bold: - "stronger reasoning capabilities" - "P2N (Previous to Next)" - "P2N adds the previous token's deep hidden state to the current token's shallow representation" - "P2N improves GSM8K accuracy by over 10 percentage points under both zero-shot and few-shot prompting, with nearly unchanged decoding latency" Is there any list? No. Is there any quote? No. Is there any code? No. Are there section headers? The text doesn't have them, so I shouldn't invent them (rule 4). I will just output one `

    ` tag. Wait, what about the repetition at the end: "...benchmarks. on general downstream benchmarks." This is clearly a typo in the original text, but rule 4 says "保留原文所有信息,不得增删改内容" (preserve all original information, no additions/deletions). So I must keep the duplicate sentence exactly as is. Let's double check rule 2: "长文用

    /

    划分小节" (Long articles use h2/h3 to divide sections). Is this a "long article"? It's a short abstract. Applying h2/h3 without explicit headers from the text might violate "不得增删改内容". I will leave it as a single `

    ` to be safe and accurate to the source. Let's refine the `` tags to ensure they don't alter the text flow and capture the most important parts: 1. "stronger reasoning capabilities" 2.