R${}^2$PO: Decoupling Rollout and Inference Policies for LLM Reasoning

Jingchu Wang, Bingbing Xu, Yige Yuan +4

Existing reinforcement learning methods for LLM reasoning implicitly assume that the policy generating training trajectories should coincide with the one producing inference respon…