Reflective Verbal Reward Design for Pluralistic Alignment

Carter Blair, Kate Larson, Edith Law

AI agents are commonly aligned with "human values" through reinforcement learning from human feedback (RLHF), where a single reward model is learned from aggregated human feedback…