首页 > AI前沿 > From Large Language Models to Efficient System One Models via Adaptive Computation

From Large Language Models to Efficient System One Models via Adaptive Computation

arXiv自然语言 2026-11-12 13:40 6 阅读 查看原文
用户现在需要把给定的新闻正文排版成语义化的HTML片段,按照规则来。首先,整个正文是一个研究相关的段落?不对,看内容,首先第一句是背景,然后是发现,然后是提出的方法,最后是实验结果。首先,先处理自然段落,这里整个是连贯的?不对,先理清楚,有没有可以划分的小节?不对,原文是一段?不对,看用户给的正文,是一段?不,等下,用户给的正文是一段连续的?不对,再仔细看:首先是背景介绍,然后发现,然后提出方法,然后实验结果。那按照规则,有没有合适的小节?哦,对了,规则说长文用h2/h3划分小节,那这里可以划分吗?等下,原文的内容结构:首先是背景和问题,然后是研究发现,然后是提出的方法,然后是实验结果。那可以划分小节吗?不过要注意不能增删改内容,所以小节标题是不是要从原文里提取?不对,规则是保留原文所有信息,不得增删改,所以不能自己加标题?哦,不对,等下看规则4,保留原文所有信息,不得增删改内容,那h2/h3的标题是不是不能自己加?那如果原文没有小标题的话,是不是就只用p?不对,再看原文的内容,整个是一段吗?等下看用户给的正文,是一段连续的文本?不对,再读: 第一句:Transferring LLMs into System One decision models typically replaces the language modeling head with a decision head while retaining the full backbone, making every decision pay for all layers. Yet it remains unclear how much depth such decisions require. 然后:We find that decision information is concentrated in intermediate layers, with early layers building essential features and the final six layers removable without loss, while independently trained smaller models largely mimic the full model rather than developing specialized decision capabilities. 然后:These findings suggest that efficiency requires not simply smaller backbones, but shared computation across different depths. 然后:Building on this, we propose \method{}, which jointly trains a full model with prefix sub-models that share its final layers and decision head through self-distillation. 然后:A lightweight gate reads the hidden state at each exit to determine whether additional computation is needed, allowing one model to operate at multiple layers. 然后:Across seven decision benchmarks, our approach matches or outperforms independently trained models at every depth, while achieving better accuracy-efficiency trade-offs through adaptive computation. 哦,不对,这些都是连贯的,但有没有必要划分小节?不对,规则说长文用h2/h3划分,那如果原文没有现成的小标题,是不是不能自己加?因为规则4说不得增删改内容。那这样的话,是不是整个作为一个p?不对,不对,再仔细看规则2:“每个自然段落用

包裹,长文用

/

划分小节”,那如果原文没有小标题,就不需要加,因为不能增内容。那接下来,有没有需要用strong的地方?重要句子,比如研究发现,或者提出的方法,或者结果?比如“We find that decision information is concentrated in intermediate layers, with early layers building essential features and the final six layers removable without loss, while independently trained smaller models largely mimic the full model rather than developing specialized decision capabilities.”这句是不是重要?还有“we propose \method{}, which jointly trains a full model with prefix sub-models that share its final layers and decision head through self-distillation.”还有“our approach matches or outperforms independently trained models at every depth, while achieving better accuracy-efficiency trade-offs through adaptive computation.”这些可以用strong吗? 不对,首先,不能增删改内容,所以strong只是标注重要,内容不变。那现在,整个原文是一段?不对,原文的几个部分是自然段落吗?不,用户给的正文是一段连续的?不对,看用户的输入,正文是连续的一段吗?等下看: 用户给的正文: Transferring LLMs into System One decision models typically replaces the language modeling head with a decision head while retaining the full backbone, making every decision pay for all layers. Yet it remains unclear how much depth such decisions require. We find that decision information is concentrated in intermediate layers, with early layers building essential features and the final six layers removable without loss, while independently trained smaller models largely mimic the full model rather than developing specialized decision capabilities. These findings suggest that efficiency requires not simply smaller backbones, but shared computation across different depths. Building on this, we propose \method{}, which jointly trains a full model with prefix sub-models that share its final layers and decision head through self-distillation. A lightweight gate reads the hidden state at each exit to determine whether additional computation is needed, allowing one model to operate at multiple layers. Across seven decision benchmarks, our approach matches or outperforms independently trained models at every depth, while achieving better accuracy-efficiency trade-offs through adaptive computation. 哦,原来这是一整段?那不对,那是不是直接用p包裹?但有没有可以划分的小节?不对,因为没有小标题,所以不能自己加,否则就是增内容了。那接下来,重要的句子用strong。比如: 第一句是背景:Transferring LLMs into System One decision models typically replaces the language modeling head with a decision head while retaining the full backbone, making every decision pay for all layers. Yet it remains unclear how much depth such decisions require. 然后研究发现:We find that decision information is concentrated in intermediate layers, with early layers building essential features and the final six layers removable without loss, while independently trained smaller models largely mimic the full model rather than developing specialized decision capabilities. 然后发现的启示:These findings suggest that efficiency requires not simply smaller backbones, but shared computation across different depths. 然后提出的方法:Building on this, we propose \method{}, which jointly trains a full model with prefix sub-models that share its final layers and decision head through self-distillation. 然后方法的细节:A lightweight gate reads the hidden state at each exit to determine whether additional computation is needed, allowing one model to operate at multiple layers. 然后实验结果:Across seven decision benchmarks, our approach matches or outperforms independently trained models at every depth, while achieving better accuracy-efficiency trade-offs through adaptive computation. 这些都是在一个p里吗?因为原文是一整段?不对,等下,用户的正文是不是一整段?是的,看起来是。那这样的话,整个用p包裹,里面的重点句子用strong。 不对,等下,再确认规则:不得增删改内容,所以不能把原文拆成多个p吗?哦,规则说每个自然段落用p包裹,那原文的这些是自然段落吗?原文给的是一段,所以就是一个自然段落?那不对,有没有可能用户给的正文虽然是一段,但可以按语义拆?不对,自然段落是原文里的段落,用户给的是一整段,所以应该用一个p。 等下,再仔细看用户的要求:“保留原文所有信息,不得增删改内容”,所以拆分段落算不算改?哦,不对,自然段落是原文里的,如果原文是一段,就不能拆成多个p,否则就是改变结构了。那现在,原文是一段,所以用

包裹整个内容,里面的重点部分用。 那现在,哪些是重点?比如研究发现、提出的方法、实验结果,这些是重要的,所以加strong。 那现在写出来:

Transferring LLMs into System One decision models typically replaces the language modeling head with a decision head while retaining the full backbone, making every decision pay for all layers. Yet it remains unclear how much depth such decisions require. We find that decision information is concentrated in intermediate layers, with early layers building essential features and the final six layers removable without loss, while independently trained smaller models largely mimic the full model rather than developing specialized decision capabilities. These findings suggest that efficiency requires not simply smaller backbones, but shared computation across different depths. Building on this, we propose \method{}, which jointly trains a full model with prefix sub-models that share its final layers and decision head through self-distillation. A lightweight gate reads the hidden state at each exit to determine whether additional computation is needed, allowing one model to operate at multiple layers. Across seven decision benchmarks, our approach matches or outperforms independently trained models at every depth, while achieving better accuracy-efficiency trade-offs through adaptive computation.

不对,等下,有没有可能划分小节?比如,原文的内容可以分为背景、研究发现、方法、实验结果,但因为原文没有小标题,所以不能自己加,否则就是增内容了,对吧?规则4说不得增删改内容,所以不能加h2/h3的标题,因为没有现成的。那这样是对的? 再检查:有没有增删改内容?所有文字都保留了,没有加,也没有删。重点句子用了strong,符合规则。每个自然段落用p包裹,这里原文是一段,所以一个p。没有用其他标签,因为没有列表、引用、代码。那这样就可以了? 等下,