SAF-OPD: Stable Advantage Fusion for On-Policy Distillation

Yifan Ding, Xincheng Wei, Yoshua Y. Li +7

Reinforcement learning with verifiable rewards (RLVR) broadcasts a single response-level reward to every token, while on-policy distillation (OPD) scores each token against a stron…