Scalable Ensembling For Mitigating Reward Overoptimisation

Ahmed M. Ahmed, Rafael Rafailov, Stepan Sharkov +2

Reinforcement Learning from Human Feedback (RLHF) has enabled significant advancements within language modeling for powerful, instruction-following models. However, the alignment o…