EPIG-Tree: Compute-Optimal Branching for Gradient-Efficient Reinforcement Learning

Nikita Khomich, Leopold Hermansson, Ido Hakimi

Reward-based reinforcement learning for language models, exemplified by Group Relative Policy Optimization (GRPO), collapses an entire stochastic trajectory into a single scalar re…