首页 > AI前沿 > Beyond AUROC: Deployment-Aware Evaluation of Model-Internal Jailbreak Detectors

Beyond AUROC: Deployment-Aware Evaluation of Model-Internal Jailbreak Detectors

arXiv自然语言 2026-11-08 09:54 12 阅读 查看原文

背景与问题

Large language models (LLMs) are increasingly deployed in user-facing systems, where jailbreak attacks can bypass safety controls and induce harmful and policy-violating outputs. Recent model-internal detectors use hidden representations, gradients, or internal signals to identify such prompts, but their practical security value depends critically on behavior at low false-positive operating points rather than on area under the receiver operating characteristic curve (AUROC) alone.

研究内容

We study two data dependencies that shape this behavior:

  • calibration data, which determine the detection threshold
  • reference data, which can influence detector scores

主要发现

  • Across four model-internal detectors, four LLMs, and twelve jailbreak families, thresholds calibrated on curated benign prompts frequently fail to preserve their false-positive rate (FPR) budgets on real-user traffic.
  • Recalibration improves false-positive control but reveals substantial family-level coverage gaps, including cases with high AUROC but low true-positive rate (TPR).
  • Causal reference-set interventions show that changing reference composition can substantially alter family-level detection and that improving coverage of one attack family can reduce coverage of others.

结论与建议

These findings show that calibration and reference data are part of a detector's security behavior. We therefore recommend evaluating model-internal jailbreak defenses using

realized FPR on deployment-like traffic and per-family TPR at explicit low-FPR operating points
, with AUROC used as a complementary ranking metric rather than the primary security measure.