One Offset, Two Directions: Baseline Adjustment via Subgroup Offsets

Jiaying Zhou, Jiachen Liu, Qingchao Chen

Group Relative Policy Optimization (GRPO) estimates advantages against the within-prompt mean reward, a baseline shared across the group. In practice, rollouts within a prompt are…