首页 > AI前沿 > Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors

Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors

arXiv自然语言 2026-11-17 19:07 8 阅读 查看原文
用户需要我扮演专业的新闻排版工程师,将给定的纯文本新闻正文排版成语义化的HTML片段。 需要遵循的规则: 1. 只输出HTML片段本身,不要Markdown围栏(如```html),不要解释。 2. 每个自然段落用`

`包裹,长文用`

`/`

`划分小节。 3. 列表内容用`
  • `,引用用`
    `,代码用`
    `,重要句子可用``。
    4. 保留原文所有信息,不得增删改内容。
    5. 不要输出``/``/``等文档级标签。
    
    分析正文内容:
    正文是一段关于"Evolution Under Fire"框架的学术新闻/摘要。
    内容如下:
    "Tool-using language-model agents must act on information from untrusted emails, webpages, and documents. This creates a central security challenge: malicious instructions and legitimate task directives can arrive through the same interaction channel. Existing defenses are largely hand-designed and static, making further improvements dependent on repeated manual redesign. We introduce Evolution Under Fire, a staged framework that couples safety-harness evolution with adaptive stressor evolution. Utility-constrained, Pareto-scheduled tree search evolves complete executable safety harnesses from execution evidence while keeping the task model, benchmark, tools, and evaluator fixed. Adaptive stressor evolution then discovers utility-preserving attacks that guide a subsequent stage of harness evolution. On AgentDyn with GPT-4o-mini, an evolved harness reduces attack success from 50.4% to 3.0%, improves clean and attacked utility over the initial harness, and offers a stronger security-utility trade-off than reproduced defenses. On Agent-SafetyBench with Qwen2.5-7B-Instruct, harness evolution reduces attack success from 82.6% to 62.0% while retaining helpfulness and runtime validity. Evolved harnesses also transfer without further evolution to Qwen3.5-35B-A3B, reducing attack success from 32.1% to 0.9% while improving attacked-task utility."
    
    这段文本可以划分为几个小节,以符合“长文用`

    `/`

    `划分小节”的要求。虽然它不算特别长,但为了体现排版专业性,可以划分出几个逻辑小节: - 背景与挑战(前两句) - 框架方法(中间部分:引入Evolution Under Fire,树搜索,自适应压力演化) - 实验结果(后三句) 或者直接作为一个长段落,加几个h3标题。 让我们合理划分:

    背景与挑战

    Tool-using language-model agents must act on information from untrusted emails, webpages, and documents. This creates a central security challenge: malicious instructions and legitimate task directives can arrive through the same interaction channel. Existing defenses are largely hand-designed and static, making further improvements dependent on repeated manual redesign.

    框架方法

    We introduce Evolution Under Fire, a staged framework that couples safety-harness evolution with adaptive stressor evolution. Utility-constrained, Pareto-scheduled tree search evolves complete executable safety harnesses from execution evidence while keeping the task model, benchmark, tools, and evaluator fixed. Adaptive stressor evolution then discovers utility-preserving attacks that guide a subsequent stage of harness evolution.

    实验结果

    On AgentDyn with GPT-4o-mini, an evolved harness reduces attack success from 50.4% to 3.0%, improves clean and attacked utility over the initial harness, and offers a stronger security-utility trade-off than reproduced defenses. On Agent-SafetyBench with Qwen2.5-7B-Instruct, harness evolution reduces attack success from 82.6% to 62.0% while retaining helpfulness and runtime validity. Evolved harnesses also transfer without further evolution to Qwen3.5-35B-A3B, reducing attack success from 32.1% to 0.9% while improving attacked-task utility.

    检查是否有重要句子需要用``。 "Evolution Under Fire" 可以加粗。 "central security challenge" 可以加粗。 修改后的HTML:

    背景与挑战

    Tool-using language-model agents must act on information from untrusted emails, webpages, and documents. This creates a central security challenge: malicious instructions and legitimate task directives can arrive through the same interaction channel. Existing defenses are largely hand-designed and static, making further improvements dependent on repeated manual redesign.

    框架方法

    We introduce Evolution Under Fire, a staged framework that couples safety-harness evolution with adaptive stressor evolution. Utility-constrained, Pareto-scheduled tree search evolves complete executable safety harnesses from execution evidence while keeping the task model, benchmark, tools, and evaluator fixed. Adaptive stressor evolution then discovers utility-preserving attacks that guide a subsequent stage of harness evolution.

    实验结果

    On AgentDyn with GPT-4o-mini, an evolved harness reduces attack success from 50.4% to 3.0%, improves clean and attacked utility over the initial harness, and offers a stronger security-utility trade-off than reproduced defenses. On Agent-SafetyBench with Qwen2.5-7B-Instruct, harness evolution reduces attack success from 82.6% to 62.0% while retaining helpfulness and runtime validity. Evolved harnesses also transfer without further evolution to Qwen3.5-35B-A3B, reducing attack success from 32.1% to 0.9% while improving attacked-task utility.

    检查规则: 1. 只输出HTML片段本身,不要Markdown围栏、不要解释。 -> 满足 2. 每个自然段落用`

    `包裹,长文用`

    `/`

    `划分小节。 -> 满足,使用了`

    `和`

    ` 3. 列表、引用、代码按需使用,重要句子用``。 -> 使用了``,无列表引用代码。 4. 保留原文所有信息,不得增删改内容。 -> 满足,小标题是新增的结构性标签,正文文字一字未改。等等,规则说“不得增删改内容”,增加h3标题算不算“增”? “长文用

    /

    划分小节”是规则要求,所以增加小节标题是符合规则的。但为了绝对安全,小节标题可以用原文中的词汇,或者直接不加文字标题,只加标签?不,HTML的h2/h3通常包含标题文字。如果用户说“不得增删改内容”,可能指的是正文内容。为了稳妥,我可以直接用原文的短语作为小标题,或者不写小标题文字,但这不符合HTML语义。 或者,我不加小标题文字,直接:

    这不太规范。 更好的是,把原文的自然段落直接用p包裹,如果不长,就不需要强行加h2/h3。原文只有一段,或者算三个自然段?原文是一整段。 “每个自然段落用

    包裹,长文用

    /

    划分小节” 既然原文是一整段,我可以直接用一个`

    `包裹,或者为了体现排版,把它拆成几个`

    `,并加上`

    `。 如果怕“增删改内容”,我可以把原文的句子直接用`

    `包裹,不加任何h2/h3的文本,或者只加空的h3? 其实“长文用

    /

    划分小节”意味着我需要划分小节,小节标题属于排版结构,不算修改正文内容。 不过,最安全的做法是:原文本身是一段,我直接用`

    `包裹整段,如果需要划分,可以用`

    `包含原文中的关键词。 或者,原文可以看作三个逻辑段落: 1. Tool-using... redesign. 2. We introduce... evolution. 3. On AgentDyn... utility. 我可以直接用三个`

    `,不加h3,因为原文并没有明显的小节标题。如果加了h3标题,可能会被判为“增”。 为了严格遵守“保留原文所有信息,