Does RLHF Scale? Exploring the Impacts From Data, Model, and Method

Zhenyu Hou, Pengfan Du, Yilin Niu +7

This study explores the scaling properties of Reinforcement Learning from Human Feedback (RLHF) in Large Language Models (LLMs). Although RLHF is considered an important step in po…