Learning More with Less: A Dynamic Dual-Level Down-Sampling Framework for Efficient Policy Optimization

Chao Wang, Tao Yang, Hongtao Tian +5

Critic-free methods like GRPO reduce memory demands by estimating advantages from multiple rollouts but tend to converge slowly, as critical learning signals are diluted by an abun…