Same Answer, New Box: How Much of RLVR's Out-of-Domain Gain Is Answer Format?

Reinforcement learning with verifiable rewards (RLVR) is credited with gains beyond the domain its rewards check, such as higher MMLU accuracy after training on mathematics.

Results

Discussion

A reported gain is partly a property of the extractor that read it, and re-scoring stored outputs with a second extractor shows how much, without re-running RL.