Consequentialist Objectives and Catastrophe

Henrik Marklund, Alex Infanger, Benjamin Van Roy

Because human preferences are too complex to codify, AIs operate with misspecified objectives. Optimizing such objectives often produces undesirable outcomes; this phenomenon is kn…