Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms

Rafael Rafailov, Yaswanth Chittepu, Ryan Park +5

Reinforcement Learning from Human Feedback (RLHF) has been crucial to the recent success of Large Language Models (LLMs), however, it is often a complex and brittle process. In the…