(Ir)rationality in AI: State of the Art, Research Challenges and Open Questions

Abstract

The concept of rationality is central to the field of artificial intelligence (AI). Whether we are seeking to simulate human reasoning, or trying to achieve bounded optimality, our goal is generally to make artificial agents as rational as possible. Despite the centrality of the concept within AI, there is no unified definition of what constitutes a rational agent. This article provides a survey of rationality and irrationality in AI, and sets out the open questions in this area. We consider how the understanding of rationality in other fields has influenced its conception within AI, in particular work in economics, philosophy and psychology. Focusing on the behaviour of artificial agents, we examine irrational behaviours that can prove to be optimal in certain scenarios. Some methods have been developed to deal with irrational agents, both in terms of identification and interaction, however work in this area remains limited. Methods that have up to now been developed for other purposes, namely adversarial scenarios, may be adapted to suit interactions with artificial agents. We further discuss the interplay between human and artificial agents, and the role that rationality plays within this interaction; many questions remain in this area, relating to potentially irrational behaviour of both humans and artificial agents.

References in corpus (9)

• 'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions
• Exploration in Deep Reinforcement Learning: A Survey
• Using cognitive psychology to understand GPT-3
• Thinking Fast and Slow in Large Language Models
• Human-Like Intuitive Behavior and Reasoning Biases Emerged in Language Models -- and Disappeared in GPT-4
• When Humans Aren't Optimal: Robots that Collaborate with Risk-Aware Humans
• (Ir)rationality and Cognitive Biases in Large Language Models
• Bias and Fairness in Computer Vision Applications of the Criminal Justice System
• Marked Attribute Bias in Natural Language Inference