Adversarial Training of Reward Models

Alexander Bukharin, Haifeng Qian, Shengyang Sun +6

Reward modeling has emerged as a promising approach for the scalable alignment of language models. However, contemporary reward models (RMs) often lack robustness, awarding high re…