OR Else: A Differentiable Trust Region for Policy Optimization

Chinmay Rane, Kanishka Tyagi, Michael Manry

PPO and the GRPO baseline studied here use clipped surrogate objectives whose favorable-direction saturation introduces an abrupt change in the scalar objective's derivative. We as…