PRPO: Aligning Process Reward with Outcome Reward in Policy Optimization

Ruiyi Ding, Yongxuan Lv, Xianhui Meng +4

Policy optimization for large language models often suffers from sparse reward signals in multi-step reasoning tasks. Critic-free methods like GRPO assign a single normalized outco…