A Systematic Analysis of Base Model Choice for Reward Modeling

Kian Ahrabian, Pegah Jandaghi, Negar Mokhberian +2

Reinforcement learning from human feedback (RLHF) and, at its core, reward modeling have become a crucial part of training powerful large language models (LLMs). One commonly overl…