Large language models are increasingly used to simulate and probe human behavior, e.g., for surveys.
For these applications, a key property is that outputs satisfy distributional alignment: to match its output distribution with some reference (in this case, human) distribution.
An obstacle in achieving distributional alignment is generalizing from small training data.
In this work, we study (1) how to distributionally align LLMs; and (2) how aligned LLMs might generalize to unseen samples and unrelated topics and tasks.
- how to distributionally align LLMs
- how aligned LLMs might generalize to unseen samples and unrelated topics and tasks
Using two large-scale benchmarks, we find that LLMs can indeed achieve distributional alignment and generalize to unseen questions of the same domain.
Cross-topic generalization depends on the relationship between topics (e.g., questions about politics are related to trust in institutions).
However, aligned LLMs can degrade in surprising ways (e.g., lose their creative ability), suggesting that its generalization properties are inherently unpredictable and unstable.