首页 > AI前沿 > Beyond the Name: Demographic Leakage in De-Identified R\'esum\'es and Evaluation Artifacts in LLM Bias Audits

Beyond the Name: Demographic Leakage in De-Identified R\'esum\'es and Evaluation Artifacts in LLM Bias Audits

arXiv自然语言 2026-09-15 09:48 2 阅读 查看原文

De-identified résumé screening assumes that redacting explicit fields prevents ethnocultural inference; however, recent audits attribute residual leakage to declared languages.

We investigate whether eliminating language fields resolves this leakage across nine open-weight models and 620 counterfactual résumés.

By holding language attributes strictly identical, we isolate unstructured prose across five ethnocultural conditions and three cue-salience tiers.

Target-group recovery averages 0.757 overall and saturates at 1.000 under high salience, demonstrating that non-language prose sustains demographic inference.

Crucially, models diverge only under faint cues (0.086-0.690), establishing salience as an essential evaluation axis.

Furthermore, pairwise LLM-as-a-judge outcomes are highly sensitive to evaluation design: forbidding ties yields an apparent selection-rate ratio of 0.39 alongside strong position and content effects, whereas permitting ties produces near-universal ties for most models ($\ge94\%$).

Downstream scoring shows only very small between-condition differences, highlighting the need to distinguish demographic signals recoverable from résumé content from effects introduced by the evaluation protocol.