It Takes 8 Tokens: Weak-to-Strong Off-Policy RL via Auxiliary Branches

Dayu Wang, Jiaye Yang, Weikang Li +4

Reinforcement learning with verifiable rewards has emerged as a standard approach for enhancing reasoning in large language models, which typically optimizes the policy by contrast…