Non-Asymptotic Global Convergence of PPO-Clip

Yin Liu, Qiming Dai, Junyu Zhang +1

Reinforcement learning (RL) has gained attention for aligning large language models (LLMs) via reinforcement learning from human feedback (RLHF). The actor-only variants of Proxima…