It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF

Taiming Lu, Lingfeng Shen, Xinyu Yang +3

Reinforcement Learning from Human Feedback (RLHF) involves training policy models (PMs) and reward models (RMs) to align language models with human preferences. Instead of focusing…