Unifying Stable Optimization and Reference Regularization in RLHF

Li He, Qiang Qu, He Zhao +4

Reinforcement Learning from Human Feedback (RLHF) has advanced alignment capabilities significantly but remains hindered by two core challenges: \textbf{reward hacking} and \textbf…