Reward Models Inherit Value Biases from Pretraining

Brian Christian, Jessica A. F. Thompson, Elle Michelle Yang +4

Reward models (RMs) are central to aligning large language models (LLMs) with human values but have received less attention than pretrained and post-trained LLMs themselves. Becaus…