首页 > AI前沿 > Model vs. User: Whose Values Prevail? Inference-Time Value Alignment in Multi-Turn Conversations

Model vs. User: Whose Values Prevail? Inference-Time Value Alignment in Multi-Turn Conversations

arXiv自然语言 2026-10-14 16:58 3 阅读 查看原文

Human values shape how we expect language models to behave.

Dominant alignment approaches treat values as global objectives, although the appropriate value orientation is personal and contextual.

Training-free alignment can steer models toward target values at inference time, yet has been largely evaluated in single-turn settings and with a single steering direction (e.g., toward safety).

We evaluate four prompt- and activation-based steering methods in multi-turn conversations, where user and assistant models push toward the same or opposing, equally desirable values.

We track alignment with a linear probe across three models and two value trade-offs.

When assistant and user pull in opposite directions, the side matching the assistant's baseline bias often prevails, and effects that look successful early often fade or reverse after a few turns.

Steering should therefore be evaluated over full conversations and in both directions of a value trade-off.