Capturing Individual Human Preferences with Reward Features

André Barreto, Vincent Dumoulin, Yiran Mao +6

Reinforcement learning from human feedback usually models preferences using a reward function that does not distinguish between people. We argue that this is unlikely to be a good…