Non-Asymptotic Global Convergence of PPO-Clip

Yin Liu, Qiming Dai, Junyu Zhang +1

Reinforcement learning has gained attention for modern Large Language Model post-training. The actor-only variants of Proximal Policy Optimization (PPO) are widely applied for thei…