MADS: Multi-Agent Dialog Simulation for Robust Conversational Agents

Task-oriented conversational agents remain fragile under realistic conditions, especially when users exhibit non-cooperative behavior.

Existing function-calling benchmarks underrepresent adversarial conversation trajectories, thereby limiting the training resources available for developing robust agents.

We present MADS (Multi-Agent Dialog Simulation)

a synthetic data generation pipeline that produces structured, diverse dialogue trajectories.

We inject adversarial conversation, producing validated conversation trajectories that capture realistic, unhappy paths.

Using MADS, we construct MADS-Bench:

a benchmark spanning six domains, 55 personas, and 1095 conversations, with approximately 66% of conversations containing failure-prone conversation scenarios.

We evaluate the robustness of the trained models using a two-stage training approach that combines supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR).