Rollout-Level Advantage-Prioritized Experience Replay for GRPO

Gyeongtae Yoo, Sanghyeok Park, Soohyuk Jang +2

Reinforcement learning from verifiable rewards with GRPO is a standard approach for post-training reasoning LLMs. It remains sample inefficient. Each rollout is used for a single g…