Personalizing AI assistants requires adapting their behavior to users' preferences and task contexts. Existing work elicits assistant personas through persona prompting or finetuning on persona-conditioned generations. However, persona descriptions alone do not identify what to change or preserve in a particular response, or which behaviors suit the current request.
Persona Learning from Feedback (PLF)
We introduce Persona Learning from Feedback (PLF), a framework that learns reusable, preference-grounded assistant personas as independent LoRA adapters on a shared frozen backbone.
- During training, a frozen self-teacher conditions on persona specifications and response-specific feedback to provide token-level supervision on each adapter's own generated prefixes.
- A feedback gate focuses corrective updates on responses judged to need improvement.
- At deployment, a supervised router uses the request or dialogue to select a single adapter or a fixed composition, without user-specific finetuning.
We evaluate PLF on single-turn personalization with model-generated critiques and multi-turn tutoring with simulated students' reactions.
Across both settings, PLF improves model-judged preference fit over fixed-persona policies and strong baselines including persona prompting and persona distillation, demonstrating how feedback-trained behaviors can be reused for adaptive assistant personalization without degrading general capabilities.