MetaRM: Shifted Distributions Alignment via Meta-Learning

Shihan Dou, Yan Liu, Enyu Zhou +9

The success of Reinforcement Learning from Human Feedback (RLHF) in language model alignment is critically dependent on the capability of the reward model (RM). However, as the tra…