To Go Far, Go Together: Diverse Preferences Induce a Curriculum for Reward Optimization

Taehyung Kim, Jongeun Choi

Learning a reward model from human feedback and optimizing a policy against it is one approach to aligning AI systems with individual users. From a fairness perspective, existing w…