Bias Fitting to Mitigate Length Bias of Reward Model in RLHF

Kangwen Zhao, Jianfeng Cai, Jinhua Zhu +5

Reinforcement Learning from Human Feedback (RLHF) relies on reward models to align large language models with human preferences. However, RLHF often suffers from reward hacking, wh…