Learning from the Right Rollouts: Data Attribution for PPO-based LLM Post-Training

Dong Shu, Denghui Zhang, Jessica Hullman

Traditional RL algorithms like Proximal Policy Optimization (PPO) typically train on the entire rollout buffer, operating under the assumption that all generated episodes provide a…