Thermodynamics as a theory of decision-making with information processing costs
arXiv:1204.6481 · doi:10.1098/rspa.2012.0683
Abstract
Perfectly rational decision-makers maximize expected utility, but crucially ignore the resource costs incurred when determining optimal actions. Here we propose an information-theoretic formalization of bounded rational decision-making where decision-makers trade off expected utility and information processing costs. Such bounded rational decision-makers can be thought of as thermodynamic machines that undergo physical state changes when they compute. Their behavior is governed by a free energy functional that trades off changes in internal energy-as a proxy for utility-and entropic changes representing computational costs induced by changing states. As a result, the bounded rational decision-making problem can be rephrased in terms of well-known concepts from statistical physics. In the limit when computational costs are ignored, the maximum expected utility principle is recovered. We discuss the relation to satisficing decision-making procedures as well as links to existing theoretical frameworks and human decision-making experiments that describe deviations from expected utility theory. Since most of the mathematical machinery can be borrowed from statistical physics, the main contribution is to axiomatically derive and interpret the thermodynamic free energy as a model of bounded rational decision-making.
26 pages, 5 figures, (under revision since February 2012)
References in corpus (3)
Cited by in corpus (47)
- Perspective: Energy Landscapes for Machine Learning
- Reward-rational (implicit) choice: A unifying formalism for reward learning
- The Two Kinds of Free Energy and the Bayesian Revolution
- Bounded rational decision-making from elementary computations that reduce uncertainty
- A bi-partite generative model framework for analyzing and simulating large scale multiple discrete-continuous travel behaviour data
- Beyond Preferences in AI Alignment
- Information-Theoretic Bounded Rationality
- Exploiting Hierarchy for Learning and Transfer in KL-regularized RL
- Generalized Thompson Sampling for Sequential Decision-Making and Causal Inference
- An Information-Theoretic Optimality Principle for Deep Reinforcement Learning
- Specialization in Hierarchical Learning Systems
- Behavior Priors for Efficient Reinforcement Learning
- Variational Bayesian Reinforcement Learning with Regret Bounds
- Free utility model for explaining the social gravity law
- Hierarchical State Abstractions for Decision-Making Problems with Computational Constraints
- The Efficiency of Human Cognition Reflects Planned Information Processing
- Bounded Rational Decision-Making with Adaptive Neural Network Priors
- LQG Control with Minimum Directed Information: Semidefinite Programming Approach
- Active Inference and Epistemic Value in Graphical Models
- Representing preorders with injective monotones
- Informational entropy thresholds as a physical mechanism to explain power-law time distributions in sequential decision-making
- A Formal Approach to Modeling the Cost of Cognitive Control
- Habit learning supported by efficiently controlled network dynamics in naive macaque monkeys
- Active Inference Tree Search in Large POMDPs
- Relative cluster entropy for power-law correlated sequences
- An Information-theoretic On-line Learning Principle for Specialization in Hierarchical Decision-Making Systems
- Moral decisions in the age of COVID-19: your choices really matter
- Hierarchical Linearly-Solvable Markov Decision Problems
- Thermodynamic fluctuation theorems govern human sensorimotor learning
- Transfer-Entropy-Regularized Markov Decision Processes
- An Information-Theoretic Perspective on Credit Assignment in Reinforcement Learning
- Reinforcement Learning with Subspaces using Free Energy Paradigm
- Hierarchical Expert Networks for Meta-Learning
- Exploring Hierarchy-Aware Inverse Reinforcement Learning
- Competing AI: How does competition feedback affect machine learning?
- Market Self-Learning of Signals, Impact and Optimal Trading: Invisible Hand Inference with Free Energy
- Jarzyski's equality and Crooks' fluctuation theorem for general Markov chains with application to decision-making systems
- Emergence of Fluctuation Relations in UNO
- Describing traveler choice behavior using the free utility model
- Bounded strategic reasoning explains crisis emergence in multi-agent market games
- Adaptive information-theoretic bounded rational decision-making with parametric priors
- A theoretical framework to explain non-Nash equilibrium strategic behavior in experimental games
- Generalized Inverse Planning: Learning Lifted non-Markovian Utility for Generalizable Task Representation
- Mixture-of-Variational-Experts for Continual Learning
- Where Do Human Heuristics Come From?
- Memory shapes time perception and intertemporal choices
- Bounded rationality for relaxing best response and mutual consistency: The Quantal Hierarchy model of decision-making