It's Not You, It's Clipping: A Soft Trust-Region via Probability Smoothing for LLM RL

Madeleine Dwyer, Adam Sobey, Adriane Chapman

Training large language models (LLMs) with reinforcement learning (RL) methods such as PPO and GRPO commonly relies on ratio clipping to stabilise updates. While effective at preve…