首页 > AI前沿 > Towards Scalable RLVR: Multimodal Instruction Following Data Synthesis and Distillation

Towards Scalable RLVR: Multimodal Instruction Following Data Synthesis and Distillation

arXiv自然语言 2026-09-13 16:50 12 阅读 查看原文

Multimodal instruction following (MMIF) is crucial for building generalist agents.

However, current training paradigms rely heavily on Supervised Fine-Tuning (SFT), which often leads to surface-level pattern matching and degrades general capabilities.

While Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising alternative, its scalability in MMIF is severely bottlenecked by the scarcity of high-quality, RL-ready multimodal data.

To bridge this gap, we present MIFS (M Multimodal Instruction Following Synthesis), a systematic pipeline designed to generate RL-ready multimodal data.

Specifically, MIFS introduces a generative constraint protocol to synthesize diverse raw samples, followed by a learnability-aware distillation mechanism that filters data based on RL training dynamics to ensure stable policy optimization.

Furthermore, a code-based verifier provides high-precision reward signals for policy learning.

The resulting dataset comprises 90k samples across 8 constraint categories and 14 task domains.

Empirical evaluations demonstrate that MIFS-trained MLLMs achieve an average improvement of 8.13% on four MMIF benchmarks and a 3× faster training convergence compared to using raw data.

Crucially, our approach mitigates the generalization trade-offs typical of SFT, preserving core visual capabilities while significantly boosting instruction-following precision.