首页 > AI前沿 > Learning When to Think: Adaptive Internal Computation for Reinforcement Learning

Learning When to Think: Adaptive Internal Computation for Reinforcement Learning

arXiv机器学习 2026-11-08 00:00 3 阅读 查看原文
我负责把给定的纯文本新闻正文排版成语义化 HTML 片段。必须严格按用户规则:只输出 HTML 片段本身,不包含 Markdown 围栏、不解释、不输出任何文档级标签,并且不能增删改原文内容。 首先,我需要理解规则:每个自然段落用 `

` 包裹;标题和新闻标题用 `

` 或 `

` 划分;如果有列表用 `
  • `;引用用 `
    `;代码用 `
    `;重要句子可以用 ``。
    
    现在看输入的正文。输入是一段学术论文摘要或介绍,内容是关于 Adaptive Internal Computation (AIC) 的。我需要仔细阅读文本,找出自然段落和小节。确认输入确实是一段纯文本新闻正文的主体内容。
    
    接下来,观察文本结构:输入是一整段,没有明显的换行或小标题。所以它应该被视为一个自然段落,用 `

    ` 包裹。 规则提到“长文用 `

    `/`

    ` 划分小节”。但这不是一个长文,只是一个段落,而且输入中没有明显的小标题。正文开头直接是正文内容,没有标题文字。所以不需要添加 `

    ` 或 `

    `。如果输入有副标题,可能需要用标题标签,但这里没有。开头是一个完整的段落。 检查原文,看是否有任何列表、引用或代码。没有列表;没有引用标记;没有代码。所以 `
      `、`
      `、`
      ` 都不适用。但有些句子可能很重要,可以用 ``。规则说“重要句子可用 ``”,不是必须。我需要判断哪些句子是重要的。原文是一篇摘要,核心贡献很多。第一句可能是关键句——描述了问题。第二句是贯穿全文的介绍主要思想。第三句是方法技术细节。之后是实验结果。
      
      仔细看,第二句可能是关键句,因为它引入了 AIC 这个主题。但同样,这只是我的专业判断,不一定需要加粗。规则说可用,所以我可以选择不加,以避免过度处理。我通常会加粗关键句以突出重点,但要保守。观察输入,这是一个单一段落。如果加粗一些部分,会不会打断文本流?可能不会。
      
      然而,我必须注意规则 4:“保留原文所有信息,不得增删改内容”。加粗本身不改动文字,只是添加标记,所以可以。但加粗的选择必须谨慎。例如,第二句“We introduce Adaptive Internal Computation (AIC), an iterative policy that learns when to stop refining its latent representation and act.” 是核心介绍,适合加粗。但第一句是问题陈述,也可能重要。
      
      为了符合专业排版标准,可能应该加粗定义 AIC 的第二句,因为它引入了基础概念。或许也可以加粗第一句,以强调问题。但我得确保不会过度。既然原文只给了一个段落,我就把整个段落用 `

      ` 包起来,并仅在有强重点的句子上加 ``。 再读一遍文本,看看哪些句子可以被强调: 1. “Iterative neural policies usually spend the same computation on every decision, although the benefit of additional processing varies across states.” —— 这是问题陈述,很关键。 2. “We introduce Adaptive Internal Computation (AIC), an iterative policy that learns when to stop refining its latent representation and act.” —— 这是主要贡献,高度关键。 3. “In AIC, we train action selection and stopping jointly, an expected-depth penalty values computation and intermediate value predictions provide auxiliary supervision.” —— 这给出了方法细节。但注意这句有点语法问题:“an expected-depth penalty values computation and intermediate value predictions provide auxiliary supervision”,可能应该是“an expected-depth penalty and values computation ...” 但原文就是这样,我不能改。它很重要,但可能不如简介句那样需要加粗。 4. “We show that when only some states benefit from further refinement, an ideal state-dependent allocator outperforms every state-independent stopping schedule at the same expected depth.” —— 这是理论结果,可能重要。 5. “With at most eight refinement steps per decision, AIC executes 3.3 steps on average across twelve benchmark tasks and lowers per-decision latency by 31--44% relative to a fixed eight-step policy on four control and robotics tasks.” —— 这是效率相关,很重要。 6. “Its mean reward exceeds the best fixed-depth policy on seven tasks and trails it on four.” —— 简要性能,可能重要。 7. “Ablating adaptive stopping and value supervision separately shows that stopping accounts for most of the reward gain on PickCube and AntMaze, where it also executes more refinement steps and that supervision accounts for most of it on RareEvent.” —— 这是消融研究,重要且详细。 结构上,没有子标题,所以只用一个 `

      `。 接下来,如果我决定加粗,我要具体定位要加粗的句子,并保证不改变任何文本。 再看文本: “In AIC, we train action selection and stopping jointly, an expected-depth penalty values computation and intermediate value predictions provide auxiliary supervision.” 这个句子里的“an expected-depth penalty values computation”可能是笔误,但必须保留原文,所以我不会修改。 还要注意,原文中“31--44%”是双连字符,不是破折号。我可以保留它。 我会把整个段落包装进 `

      `,然后在选定的句子外面加上 ``。 例如:

      Iterative neural policies usually spend the same computation on every decision, although the benefit of additional processing varies across states. We introduce Adaptive Internal Computation (AIC), an iterative policy that learns when to stop refining its latent representation and act. In AIC, we train action selection and stopping jointly, an expected-depth penalty values computation and intermediate value predictions provide auxiliary supervision. We show that when only some states benefit from further refinement, an ideal state-dependent allocator outperforms every state-independent stopping schedule at the same expected depth. With at most eight refinement steps per decision, AIC executes 3.3 steps on average across twelve benchmark tasks and lowers per-decision latency by 31--44% relative to a fixed eight-step policy on four control and robotics tasks. Its mean reward exceeds the best fixed-depth policy on seven tasks and trails it on four. Ablating adaptive stopping and value supervision separately shows that stopping accounts for most of the reward gain on PickCube and AntMaze, where it also executes more refinement steps and that supervision accounts for most of it on RareEvent.

      这样可以吗?我强调了第二句和第五句。可能还应该加粗第一句,因为它设定了背景。试试:

      Iterative neural policies usually spend the same computation on every decision, although the benefit of additional processing varies across states. We introduce Adaptive Internal Computation (AIC), an iterative policy that learns when to stop refining its latent representation and act. In AIC, we train action selection and stopping jointly, an expected-depth penalty values computation and intermediate value predictions provide auxiliary supervision. We show that when only some states benefit from further refinement, an ideal state-dependent allocator outperforms every state-independent stopping schedule at the same expected depth. With at most eight refinement steps per decision, AIC executes 3.3 steps on average across twelve benchmark tasks and lowers per-decision latency by 31--44% relative to a fixed eight-step policy on four control and robotics tasks. Its mean reward exceeds the best fixed-depth policy on seven tasks and trails it on four. Ablating adaptive stopping and value supervision separately shows that stopping accounts for most of the reward gain on PickCube and AntMaze, where it also executes more refinement steps and that supervision accounts for most of it on RareEvent.

      这可能太多了,但技术上没有增删改内容。用户要求重要句子可用 ``,所以我可以选择强调。我应该决定哪些真正重要。第一句是问题陈述,第二句是贡献,第五句是关键效率结果,最后一句是消融。所以这些强调是合理的。 或许我还需要加粗最后一句,因为它是对关键组件的消融分析。现在先加上。 最终输出应该是一个 HTML 片段。我会确保只输出片段,没有额外文本。 还需要检查是否遗漏了任何标点或内容。例如,原文是“31--44%”,我原样保留。原文中“an expected-depth penalty values computation”放在逗号后,没有“and”连接动作,但原文就是这样,所以不能动。 现在确认没有其他结构。没有列表、没有