Swap, Replicate, Read the Mass: A Protocol for Auditing Demographic Bias in System One Decision Models

1. 分析请求： * 角色：专业的新闻排版工程师。 * 任务：将提供的纯文本新闻正文排版成语义化的HTML片段。 * 规则： 1. 只输出HTML片段本身，不要Markdown围栏，不要解释。 2. 每个自然段落用`

`包裹，长文用`

`/`

`划分小节（视情况而定，这里文本不长，但为了结构清晰可以加入标题或直接按段落）。 3. 列表用`

• `，引用用`

  `，代码用`

  `，重要句子用``。
          4.  保留原文所有信息，不得增删改内容。
          5.  不要输出``/``/``等文档级标签。
      *   输入文本：一段关于System One决策模型偏见审计的英文新闻/论文摘要。
  
  2.  解析输入文本并划分结构：
      *   原文：
          System One decision models replace text generation with typed, probability-scored answers and are entering production. Answers execute without a person in the loop, and no multi-model audit exists on established bias benchmarks. We provide that audit: three benchmarks: BBQ (all eleven categories), CrowS-Pairs, and SilverSpoon, adapted to the typed interface, twelve models plus a no-thinking ablation, and 1,638,000 decisions. Every order-sensitive item runs twice with options swapped; native typed models also swap labels; ten configurations repeat under extra seeds and batch 16. We find widespread stereotype alignment, order-dependent answers, and confidence that tracks instability rather than bias. In BBQ, 97 of 143 model-by-category scores are significant and 94 aligned, while the commercial reference abstains on 99.5% of ambiguous items, hiding aligned probability mass. Option swaps flip 0.0% to 72.6% of choices; typed scoring stays below 10% except Laya (22-33%), and untuned letter prompting reaches 73%. CrowS-Pairs preference spans 0.496 to 0.816, and boolean answers can be decided by position (1.00 versus 0.005). SilverSpoon judgments shift toward lower-SES labels in five of eight decision models, inherited from the 9B backbone or induced by tuning. Audits must permute options and reverse questions by default; order-invariant scoring should become a procurement requirement.
      *   结构分析：
          *   这是一段连贯的摘要，但可以根据逻辑拆分为几个自然段，以符合“每个自然段落用
  包裹”的要求。
          *   段落1（背景与提出的问题）：System One decision models... no multi-model audit exists on established bias benchmarks.
          *   段落2（审计方法）：We provide that audit: ... batch 16.
          *   段落3（主要发现）：We find widespread stereotype alignment... rather than bias.
          *   段落4（具体基准结果 - BBQ）：In BBQ... hiding aligned probability mass.
          *   段落5（具体基准结果 - 顺序与CrowS-Pairs）：Option swaps flip... versus 0.005).
          *   段落6（具体基准结果 - SilverSpoon）：SilverSpoon judgments... induced by tuning.
          *   段落7（结论与建议）：Audits must permute options... procurement requirement.
      *   或者，把整个当作几段：
          *   `
  `背景与审计方法
   (可以加小标题以符合“长文用h2/h3划分小节”，虽然这里不长，但加上更符合“语义化”和规则2的精神)
          *   `
  System One decision models...
  `
          *   `
  We provide that audit: three benchmarks: 
  BBQ...
  CrowS-Pairs...
  SilverSpoon...
  , twelve models...
  ` (注意，原文是 "three benchmarks: BBQ (all eleven categories), CrowS-Pairs, and SilverSpoon, adapted to..."，如果用列表，要小心不改变原意。原文是逗号分隔的列表。用 `
  ` 可能会改变句子结构，不如直接在文本中用 `` 标记重要部分，或者将列表提取出来。)
          *   让我们仔细看原文的列表部分："three benchmarks: BBQ (all eleven categories), CrowS-Pairs, and SilverSpoon"。可以排版为：
              `
  We provide that audit: three benchmarks: 
  BBQ (all eleven categories)
  CrowS-Pairs
  SilverSpoon
  , adapted to the typed interface, twelve models plus a no-thinking ablation, and 1,638,000 decisions.
  ` (但HTML中`
  `直接嵌在句子中间不太符合HTML语法规范，最好放在块级元素中，或者用`
  
             `/` 
             
  
              `包裹列表项，如果严格不改变内容，可以用``或者直接文本加``。) 其实，规则说“列表内容用` 
                
  
                 
  `”，我可以把这三个基准做成列表。 例如： 
                  
  We provide that audit: three benchmarks:
                  
  
                   
  BBQ (all eleven categories)
                   
  CrowS-Pairs
                   
  SilverSpoon
                  
                  
  , adapted to the typed interface...
   (这样断开了句子，改变了原意/结构)。 更好的做法： 
                  
  We provide that audit: three benchmarks, adapted to the typed interface, twelve models plus a no-thinking ablation, and 1,638,000 decisions:
                  
  
                   
  BBQ (all eleven categories)
                   
  CrowS-Pairs
                   
  SilverSpoon
                  
                  
   (这稍微调整了标点，但规则说“不得增删改内容”。那就老老实实用文本，或者把逗号列表用``标出，或者用` 
                   
  
                    `包裹这三个词，但保留上下文。) 或者： 
                    
  We provide that audit: three benchmarks: BBQ (all eleven categories), CrowS-Pairs, and SilverSpoon, adapted to the typed interface, twelve models plus a no-thinking ablation, and 1,638,000 decisions.
   规则要求“列表内容用` 
                    
  
                     
  `”，如果原文有列表特征（如A, B, and C），我可以把它拆成列表，但这会破坏句子的连贯性。为了严格遵守“不得增删改内容”，最好保持句子完整，只把关键术语加粗。不过，如果我把“BBQ..., CrowS-Pairs, and SilverSpoon”做成列表，确实改变了语法结构。 折中方案：把整个句子保留，对重要名词用``。 或者，把实验设置部分用` 
                       
  `划分。 * 重新划分段落和标题：
                       
  Background and Audit Methodology
                       
  System One decision models replace text generation with typed, probability-scored answers and are entering production. Answers execute without a person in the loop, and no multi-model audit exists on established bias benchmarks. We provide that audit: three benchmarks: BBQ (all eleven categories), CrowS-Pairs, and SilverSpoon, adapted to the typed interface, twelve models plus a no-thinking ablation, and 1,638,000 decisions. Every order-sensitive item runs twice with options swapped; native typed models also swap labels; ten configurations repeat under extra seeds and batch 16.
                       
  Key Findings
                       
  We find widespread stereotype alignment, order-dependent answers, and confidence that tracks instability rather than bias.
                       
  Benchmark Results
                       
  In BBQ, 97 of 143 model-by-category scores are significant and 94 aligned