Misalignment from Treating Means as Ends

Henrik Marklund, Alex Infanger, Benjamin Van Roy

Reward functions, learned or manually specified, are rarely perfect. Instead of accurately expressing human goals, these reward functions are often distorted by human beliefs about…