SFT Overtraining Predicts Rank Inversion via Entropy Collapse Under RLVR

Siddharth Aphale, Kelly Liu

The standard heuristic of selecting the SFT checkpoint with the highest pass@1 for GRPO can fail when SFT compresses the rollout distribution. For binary rewards, the expected with…