ANO: A Principled Approach to Robust Policy Optimization

Yiheng Zhang, Yiming Wang, Kaiyan Zhao +3

Proximal Policy Optimization (PPO) dominates reinforcement learning and LLM alignment but relies on a "hard clipping" mechanism that discards valuable gradients. Conversely, uncons…