首页 > AI前沿 > What RL on Expert-Built Tasks Teaches Nemotron 3.5 Super

What RL on Expert-Built Tasks Teaches Nemotron 3.5 Super

arXiv自然语言 2026-11-18 13:00 19 阅读 查看原文
用户现在需要把给定的英文新闻正文排版成语义化的HTML片段,按照规则来。首先,看正文内容,首先第一部分是讲训练的基础,然后是小节?不对,先理清楚内容。首先,开头的段落是讲训练的对象和方式,然后接下来是基准测试的结果,然后是适配的行为表现?不对,先看自然段落,原文其实是一个长段落?不对,仔细看,原文的段落:首先第一句是We post-train...然后接下来的几个句子,然后是基准测试的结果,然后是适配的行为部分。不对,需要划分小节吗?哦,规则说长文用h2/h3划分小节。那首先,整体可以分几个小节?比如第一个小节是训练设置,然后是基准性能,然后是适配行为?不对,不对,要保留所有信息,不能增删。首先,先看原文的结构: 第一部分:训练方法:对Nemotron 3.5 Super用RL在五个专家数据集上逐个训练,每个用Tinker上的CISPO训练新的rank-32 LoRA适配器,无监督阶段,每个适配器和基础模型在相同任务、测试工具、设置下对比。 第二部分:基准测试结果:12个头部基准,8个其他团队的4个Surge AI的,提升范围从IFBench的+4.5到GDP_pdf的+13.8,然后列举各个基准的提升,还有文档适配器在AutomationBench的+8.0。 第三部分:行为表现:在和基础模型并排看的尝试中,基础模型的失败模式重复,每个适配器更常做到基础模型没做到的,然后列举各个适配器的行为。 那现在需要把这些用h2划分小节吗?首先,开头可以用一个h2讲训练设置,然后h2讲基准性能,然后h2讲适配行为表现?不对,要准确,不能自己加内容,但是小节标题是概括性的?不对,规则说保留原文所有信息,不得增删改内容。哦,等一下,h2/h3是用来划分小节的,但是标题的内容?不对,原文没有标题,那我们的h2是概括小节的内容,但是不能添加原文没有的信息?不对,再看规则:保留原文所有信息,不得增删改内容。那原文的所有内容都要在p、li这些里面,h2/h3是小节的划分,但是标题应该是对小节内容的概括,但是不能改变内容。或者,是不是原文的结构里的部分,我们用h2来分,比如: 首先,

训练设置

?不对,会不会算增删?不对,规则里的长文用h2/h3划分小节,是结构上的划分,标题是必要的结构,但是原文的内容都在段落里。哦,不对,等一下,原文的所有信息都要保留,不能增删,所以原文的每一句话都要在HTML的标签里,h2/h3是结构标签,用来划分小节,但是内容还是原文的,不能加额外的文字到h2里?不对,不对,再仔细看用户的要求:3. 每个自然段落用

包裹,长文用

/

划分小节。哦,对,长文的话用h2/h3划分小节,也就是把长文分成几个小节,每个小节用h2标题?但是标题的内容是什么?不对,原文没有标题,那是不是不需要自己加标题?不对,不对,再想,原文的内容其实是三个部分:1. 训练的配置和方法;2. 基准测试的结果;3. 适配器的行为表现。那我们可以给每个小节一个合适的h2,但是这些h2是对内容的概括,不算增删信息,因为是结构划分。然后,里面的内容用p包裹。 等一下,再仔细看原文的内容,不要拆分错。首先,原文的第一部分:“We post-train NVIDIA's Nemotron 3.5 Super (120B total, 12B active parameters) with reinforcement learning (RL) on five expert-built task datasets, one at a time: real-world software engineering, complex constraints, long-horizon tool use, professional chart understanding and enterprise document reasoning. Each run trains a fresh rank-32 LoRA adapter with CISPO on Tinker, with no supervised stage, and each adapter is compared with the base model on the same tasks, harness and settings.” 这部分是训练的设置,所以可以用

Training Setup

?不对,不对,原文是英文,所以标题也用英文,对,因为原文是英文的,所以h2用英文的概括,准确,不改变内容。然后这部分内容放在p里。 然后第二部分:“On twelve headline benchmarks, eight built by other groups and four by Surge AI, the gains range from +4.5 points on IFBench to +13.8 on GDP_pdf, which rises from 10.4% to 24.2%; SWE-bench Pro V2 rises from 86.9% to 92.1%, Terminal-Bench 2.1 from 63.9% to 70.3%, MultiChallenge from 58.3% to 66.6%, AutomationBench from 14.8% to 27.5% and Chartography from 39.4% to 50.6%. The document adapter, trained without API tools, also gains +8.0 points on AutomationBench.” 这部分是基准性能,所以

Benchmark Performance

,然后内容放在p里。 然后第三部分:“In attempts read side by side with the base model's on tasks each adapter newly solves, the base model's failures recur in a few modes, and each adapter does more often what the base model missed: the tool-use adapter finds the data and the rules before acting, the coding adapter writes its own test or scratch files on SWE-bench Pro V2, the constraints adapter carries earlier turns into its answers, the chart adapter measures the marks in code, and the document adapter reads AutomationBench's spreadsheets.” 这部分是适配器的行为,所以

Adapter Behavior

,然后这里后面的列举可以用ul li,因为有多个并列的内容,规则说列表内容用ul li。哦,对,原文里冒号后面是五个并列的适配器的行为,所以可以把这些做成列表,这样更语义化。 现在整理一下: 首先

Training Setup

,然后

We post-train NVIDIA's Nemotron 3.5 Super (120B total, 12B active parameters) with reinforcement learning (RL) on five expert-built task datasets, one at a time: real-world software engineering, complex constraints, long-horizon tool use, professional chart understanding and enterprise document reasoning. Each run trains a fresh rank-32 LoRA adapter with CISPO on Tinker, with no supervised stage, and each adapter is compared with the base model on the same tasks, harness and settings.

然后

Benchmark Performance

,然后

On twelve headline benchmarks, eight built by other groups and four by Surge AI, the gains range from +4.5 points on IFBench to +13.8 on GDP_pdf, which rises from 10.4% to 24.2%; SWE-bench Pro V2 rises from 86.9% to 92.1%, Terminal-Bench 2.1 from 63.9% to 70.3%, MultiChallenge from 58.3% to 66.6%, AutomationBench from 14.8% to 27.5% and Chartography from 39.4% to 50.6%. The document adapter, trained without API tools, also gains +8.0 points on AutomationBench.

这里重要的句子,比如提升的数值可以用strong,符合规则。 然后

Adapter Behavior

,然后

In attempts read side by side with the base model's on tasks each adapter newly solves, the base model's failures recur in a few modes, and each adapter does more often what the base model missed:

然后
  • the tool-use adapter finds the data and the rules before acting
  • the coding adapter writes its own test or scratch files on SWE-bench Pro V2
  • the constraints adapter carries earlier turns into its answers
  • the chart adapter measures the marks in code
  • the document adapter reads AutomationBench's spreadsheets
等一下,