Headroom-Drift Replay: A Primitive for Principled Replay Control in GRPO

Hyun Bin Park, Du-Seong Chang

RL-based post-training for reasoning models is increasingly bottlenecked by repeated fresh rollout generation, particularly in agentic settings where environment interaction domina…