Controlled Symbolic Inputs and Model Decisions
Can controlled symbolic inputs reveal attention heads that influence a language model's decisions from natural language evidence? We introduce controlled evidence aggregation (CEA), which varies the counts and order of two symbols in otherwise fixed prompts and measures changes in a frozen Transformer's preference between two answer tokens.
The preference is measured by their logit difference; the prompts prescribe no correct answer. Balanced utility causal selection (BUC) identifies compact head sets by jointly evaluating how much ablating their outputs removes the count effect and how closely patching mean outputs between count conditions recovers the effect in both directions.
Bounds and Finite Sample Recovery
We derive conditional bounds linking head attribution to intervention effects and specifying when the attribution shortlist can be recovered from finite samples.
Results
Symbolic Ablation Across Orderings
- In Pythia-410M, ablating sets of 1-12 heads selected separately for four ordering regimes removes 65.5-91.6% of the count effect.
- The sets retain their influence on count comparisons not used for selection.
Natural Language Task Transfer
To test relevance beyond symbolic inputs, we evaluate a natural language task asking which state is supported by more reports. Ablating eight heads selected only on symbolic yes/no prompts lowers mean accuracy by 8.2 percentage points, versus 0.8 for random sets matched by size and layer.
Our results demonstrate that controlled symbolic probes can identify compact attention head sets whose causal influence extends to natural language decisions.