Towards Better Training Signal: Advantage Clipped Policy Optimization

Ruichuan Huang, Jinghan Liu, Congliang Chen

Reinforcement learning (RL) has become a cornerstone for improving the reasoning capabilities of large language models (LLMs), but the need for on-policy data substantially limits…