Occam's razor is insufficient to infer the preferences of irrational agents

Stuart Armstrong, Sören Mindermann

Inverse reinforcement learning (IRL) attempts to infer human rewards or preferences from observed behavior. Since human planning systematically deviates from rationality, several a…