Beyond Entropy: Correctness-Aware Advantage Shaping via Contrastive Policy Optimization

Weiwen Xu, Jia Liu, Hou Pong Chan +4

Reinforcement learning with verifiable rewards (RLVR) commonly uses entropy for advantage shaping. However, entropy cannot distinguish useful uncertainty from detrimental confusion…