Automated interpretability scores an explanation of a sparse-autoencoder (SAE) feature by asking a model to predict, token by token, where the feature fires on evaluation examples or to judge whether marked tokens fit it. We find that the evaluation data standard pipelines build can be scored without reading the explanation.
On Neuronpedia's stored fuzzing runs for 500 random GPT-2 small features, 91.3% of activating examples are cropped with the peak on the centre token, and a rule that says "yes" when the centre token is marked reaches balanced accuracy 0.979, against 0.838 for the stored LLM scorers (recomputed from the stored per-item verdicts, on marks rebuilt from the pipeline's code); a rule reading only the mark count reaches 0.969.
- 91.3% of activating examples are cropped with the peak on the centre token
- A rule that says "yes" when the centre token is marked reaches balanced accuracy 0.979, against 0.838 for the stored LLM scorers (recomputed from the stored per-item verdicts, on marks rebuilt from the pipeline's code)
- A rule reading only the mark count reaches 0.969
The cue changes form across pipelines: current delphi always marks the same slot in its fuzzing negatives, and full-context runs mark negatives with more tokens.
OpenAI's simulation benchmark lists each neuron's top-activating sequences first; a rule that predicts activity on the first five scores 0.227 in pooled correlation against 0.160 for GPT-4's published simulator (post hoc).
We give closed forms for what such rules can score;
- show that one scorer's simulation scores are higher on stored snippets than on natural windows but not shown to be reordered
- build windows that remove the position cue
- derive but do not build negatives that remove the mark cues
- separate the sequence cue by decomposition
On natural-text windows cut at random offsets, the best fixed-offset prior shared by all features scores 0.015.
On them, that scorer, a typed decision model (Jev), reaches a pooled correlation of 0.331 over 49 features, against 0.013