首页 > AI前沿 > Relation Before Entity: Deferred Commitment in Language Model Factual Recall

Relation Before Entity: Deferred Commitment in Language Model Factual Recall

arXiv自然语言 2026-07-14 06:34 5 阅读 查看原文

We ask whether relation-type information (e.g., capital-of) and entity-specific information (e.g., France to Paris) become causally active at the final-token position at the same depth during recall.

Using four complementary causal diagnostics across four decoder-only models and eight prompt families, we find a robust temporal asymmetry: relation information becomes generation-controlling before entity information does.

Relation onset precedes entity onset by 10-16 tested layers (31-44% of network depth) at threshold 0.4, with the ordering holding across all 16 model-threshold combinations for thresholds 0.2-0.5.

Critically, entity information is not absent early: entity-token patching succeeds at 90-100% in early layers.

Instead, entity commitment to generation is deferred: entity information is available at the entity-token position but becomes generation-controlling at the final token only after being routed there.