首页 > AI前沿 > TRACER: Trace-Regularized Training Against Indirect Prompt Injection

TRACER: Trace-Regularized Training Against Indirect Prompt Injection

arXiv机器学习 2026-10-14 04:03 3 阅读 查看原文

Indirect prompt injection uses untrusted retrieved content, tool outputs, or agent messages to override an LLM's intended behavior.

We show that defense robustness is channel-conditioned: the same attack succeeds 18% of the time as a follow-up message but 53% when placed in retrieved context.

Proposed Method

We propose TRACER (Trace-Regularized Adversarial Channel Exposure with Reward-shaping), a channel-aware adversarial training method that samples presentation wrappers and combines an answer-anchored, evidence-gated reward with ATRC, which aligns attacked reasoning with clean evidence when an injection is invalid.

Experimental Results

On GSM8K with Qwen2.5-3B-Instruct, TRACER reduces RAG-framed ASR from 65.5% to 6.1% while preserving 78-82% clean accuracy.

Joint RAG+tool training achieves 13.9% average ASR across framings at 3B scale, while a single epoch at 7B reaches 9.4%.

Conclusion

These results demonstrate that robust defenses must explicitly train and evaluate across deployment channels.