首页 > AI前沿 > One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents

One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents

arXiv自然语言 2026-09-20 13:58 4 阅读 查看原文
软件工程 强化学习 智能体
1. 需求分析：用户需要将提供的纯文本新闻正文转换为语义化的HTML片段。角色设定为专业的新闻排版工程师。 2. 约束条件检查： - 只输出HTML片段本身。 - 不要Markdown围栏（即不要```html）。 - 不要解释（即不要“好的，这是您的HTML”或“以下是排版结果”等废话）。 - 每个自然段落用`

`包裹。 - 长文用`

`/`

`划分小节（这段文本虽然不长，但可以分为几个逻辑部分：背景与动机、框架与方法、实验结果）。 - 列表内容用`
  • `（原文中“Refresh-Repair-Expand (RRE)”的具体内容可以用列表展示）。 - 引用用`
    `（原文没有明显的引用）。 - 代码用` 
    `（原文没有代码）。
   - 重要句子可用``（例如核心方法名称、关键数据结果）。
   - 保留原文所有信息，不得增删改内容（这是最关键的约束，不能添加小标题的文字到正文中，不能漏掉任何句子，小标题只能作为结构标签存在，或者干脆不加文字只加标签？不，规则说“用
    /
    划分小节”，通常意味着可以添加小标题，但为了严格遵守“不得增删改内容”，最好不加文字小标题，或者只在HTML标签内不加文字。等等，如果我加了文字小标题，算不算“增删改内容”？为了绝对安全，我使用`
    `和`
    `标签包裹原有的段落开头，或者干脆不添加任何非原文的文字。最稳妥的做法是：将原文分成几个段落，用`
    `或`
    `包裹第一句，或者直接把原文的几句话用`
    `包裹，用`
    `做无文字的分节（这在HTML中不规范）。
   - 重新思考“长文用
    /
    划分小节”：如果原文没有标题，我强行加标题（如“背景”、“方法”），就违反了“不得增删改内容”。因此，我可以把原文的某些句子作为小节标题，或者不使用文字，只利用标签进行结构化。但HTML的`
    `通常包含文字。如果我把第一句话变成`
    `，算不算修改？
   - 让我们看原文：“Repository-level software engineering (SWE) comprises...” 这是一句话。
   - 如果我这样排版：
     ```html
     
    Repository-level software engineering (SWE) comprises heterogeneous task categories, whose progress under pooled agentic reinforcement learning can be uneven: gains in some categories coincide with regressions in others, while aggregate resolution obscures these changes.
    
     
    Motivated by this category see-saw, we develop a category-aware expert-training and policy-integration framework. Executable task construction and SWE Labeler, an evidence-grounded multi-axis labeling system, organize the training pools.
    
     ```
     这样算不算“增删改”？把句子变成标题，改变了语义层级。
   - 更安全的做法：不添加任何非原文的文字，将原文按逻辑切分为几个`
    `，如果需要`
    /
    `，可以把原文中的关键术语用`
    `包裹，或者干脆只用`
    `，因为规则说“长文用
    /
    划分小节”，这段文本并不算“长文”（只有几段）。但为了体现对规则的遵循，我可以把第一句话或某个核心概念用`
    `或`
    `包裹，不添加额外文字。
   - 或者，原文中“Refresh-Repair-Expand (RRE)”后面的解释可以用列表。
   - 让我们仔细切分原文：
     原文1: "Repository-level software engineering (SWE) comprises heterogeneous task categories, whose progress under pooled agentic reinforcement learning can be uneven: gains in some categories coincide with regressions in others, while aggregate resolution obscures these changes."
     原文2: "Motivated by this category see-saw, we develop a category-aware expert-training and policy-integration framework."
     原文3: "Executable task construction and SWE Labeler, an evidence-grounded multi-axis labeling system, organize the training pools."
     原文4: "Initial category-specific RL improves average training success while leaving uneven instance-level progress, motivating explicit consolidation of successful behavior and policy-adaptive task selection."
     原文5: "Same-origin category experts alternate long-horizon Agentic-miniRL with Refresh-Repair-Expand (RRE): the updated policy refreshes instance mastery, reuses its own verified successful trajectories for Repair SFT, and reselects tasks for further RL."
     原文6: "Label-routed multi-teacher on-policy distillation (MOPD) consolidates the experts into one deployable student, with ReLU-gated reward extrapolation keeping only each teacher's improving direction over the reference."
     原文7: "Expert training and policy integration require no external model to provide solution trajectories or action targets."
     原文8: "On a fixed, audit-filtered Pro-618 subset, we connect Pooled RL and Balanced RL, expert development, and single-model integration through overall and per-category resolution and expert-gain recovery."
     原文9: "In final-model evaluations, MOPD achieves mean resolution of 86.19% on SWE-bench Pro V2 and 71.78% on SWE-bench Multilingual, exceeding the base model by 4.57 and 3.89 percentage points, respectively."
     原文10: "The same three 27B experts also improve a Qwen3.6-35B-A3B student across the evaluated task sets."

   - 如何划分小节？
     小节1：背景与动机 (原文1, 2) -> 如果我加标题“背景与动机”，就违反了“不得增删改内容”。所以我只能把原文1用`
    `包裹，或者用`
    `。
     其实，规则说“长文用
    /
    划分小节”，这段不长，我可以全部用`
    `。但为了展示能力，我可以把核心结论或关键方法名用`
    `包裹。
     比如：
     ```html
     
    Repository-level software engineering (SWE) comprises heterogeneous task categories, whose progress under pooled agentic reinforcement learning can be uneven: gains in some categories coincide with regressions in others, while aggregate resolution obscures these changes. Motivated by this category see-saw, we develop a category-aware expert-training and policy-integration framework.
    
     
     
    Executable task construction and SWE Labeler, an evidence-grounded multi-axis labeling system, organize the training pools. Initial category-specific RL improves average training success while leaving uneven instance-level progress, motivating explicit consolidation of successful behavior and policy-adaptive task selection.
    
     
     
    Refresh-Repair-Expand (RRE)
    
     
    Same-origin category experts alternate long-horizon Agentic-miniRL with Refresh-Repair-Expand (RRE): the updated policy refreshes instance mastery, reuses its own verified successful trajectories for Repair SFT, and reselects tasks for further RL.
    
     ```
     等等，把“Refresh-Repair-Expand (RRE)”单独提出来当`
    `，算不算修改了原文的连贯性？原文是“Same-origin category experts alternate long-horizon Agentic-miniRL with Refresh-Repair-Expand (RRE): the updated policy...”。如果我把它拆成标题和段落，就改变了原文的句子结构。
     为了严格遵守“不得增删改内容”，我不能拆分原有的句子，也不能添加任何非原文的文字。
     那么，我只能把完整的句子或完整的段落用`
    `包裹。对于`
    /
    `，我可以包裹完整的第一句话或某个完整的句子，作为小节标题。
     例如：
     ```html
     
    Repository-level software engineering (SWE) comprises heterogeneous task categories, whose progress under pooled agentic reinforcement learning can be uneven: gains in some categories coincide with regressions in others, while aggregate resolution obscures these changes.
    
     
    Motivated by this category see-saw, we develop a category

相关推荐