ACPO: Asymmetric Credit Policy Optimization via Mode-Local Entropy Surrogate

Zijun Xie, Yuyang You, Yongzhi Li +8

Outcome-supervised reinforcement learning scales to verifiable reasoning tasks, but trajectory-level rewards assign the same outcome signal to all sampled tokens, overlooking their…