Information-Theoretic Bounded Rationality

Abstract

Bounded rationality, that is, decision-making and planning under resource limitations, is widely regarded as an important open problem in artificial intelligence, reinforcement learning, computational neuroscience and economics. This paper offers a consolidated presentation of a theory of bounded rationality based on information-theoretic ideas. We provide a conceptual justification for using the free energy functional as the objective function for characterizing bounded-rational decisions. This functional possesses three crucial properties: it controls the size of the solution space; it has Monte Carlo planners that are exact, yet bypass the need for exhaustive search; and it captures model uncertainty arising from lack of evidence or from interacting with other agents having unknown intentions. We discuss the single-step decision-making case, and show how to extend it to sequential decisions using equivalence transformations. This extension yields a very general class of decision problems that encompass classical decision rules (e.g. EXPECTIMAX and MINIMAX) as limit cases, as well as trust- and risk-sensitive planning.

47 pages, 19 figures

References in corpus (4)

• Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning
• LQG Control with Minimum Directed Information: Semidefinite Programming Approach
• Information Theory - The Bridge Connecting Bounded Rational Game Theory and Statistical Physics
• Bounded Planning in Passive POMDPs

Cited by in corpus (7)

• Variational Bayesian Reinforcement Learning with Regret Bounds
• The Efficiency of Human Cognition Reflects Planned Information Processing
• Bounded Rational Decision-Making with Adaptive Neural Network Priors
• A Tutorial on Sparse Gaussian Processes and Variational Inference
• Fully Decentralized Policies for Multi-Agent Systems: An Information Theoretic Approach
• Market Self-Learning of Signals, Impact and Optimal Trading: Invisible Hand Inference with Free Energy
• Bounded rationality for relaxing best response and mutual consistency: The Quantal Hierarchy model of decision-making