The thermodynamics of prediction

Abstract

A system responding to a stochastic driving signal can be interpreted as computing, by means of its dynamics, an implicit model of the environmental variables. The system's state retains information about past environmental fluctuations, and a fraction of this information is predictive of future ones. The remaining nonpredictive information reflects model complexity that does not improve predictive power, and thus represents the ineffectiveness of the model. We expose the fundamental equivalence between this model inefficiency and thermodynamic inefficiency, measured by dissipation. Our results hold arbitrarily far from thermodynamic equilibrium and are applicable to a wide range of systems, including biomolecular machines. They highlight a profound connection between the effective use of information and efficient thermodynamic operation: any system constructed to keep memory about its environment and to operate with maximal energetic efficiency has to be predictive.

arXiv comment5 pages, 1 figure

References in corpus (7)

• A nonequilibrium equality for free energy differences
• The Entropy Production Fluctuation Theorem and the Nonequilibrium Work Relation for Free Energy Differences
• Verification of the Crooks fluctuation theorem and recovery of RNA folding free energies
• Work and heat probability distribution of an optically driven Brownian particle: Theory and experiments
• Information theoretic approach to interactive learning
• Beyond Boltzmann-Gibbs statistics: Maximum entropy hyperensembles out-of-equilibrium
• Near-equilibrium measurements of nonequilibrium free energy

Cited by in corpus (103)

• Thermodynamics with continuous information flow
• Information processing and the second law of thermodynamics: an inclusive, Hamiltonian approach
• Thermodynamic costs of information processing in sensory adaption
• Efficiency of cellular information processing
• Information Thermodynamics on Causal Networks
• The free energy principle made simpler but not too simple
• Stochastic thermodynamics of computation
• The geometry of thermodynamic control
• Theory of nonequilibrium free energy transduction by molecular machines
• Stochastic thermodynamic interpretation of information geometry
• Thermodynamic and Logical Reversibilities Revisited
• Quantum work and the thermodynamic cost of quantum measurements
• On Bayesian Mechanics: A Physics of and by Beliefs
• Stochastic time-evolution, information geometry and the Cramer-Rao Bound
• Thermodynamics of statistical inference by cells
• Role of Mutual Information in Entropy Production under Information Exchanges
• Pre-Cooling Strategy Allows Exponentially Faster Heating
• Stochastic thermodynamics with information reservoirs
• Sensory capacity: an information theoretical measure of the performance of a sensor
• The thermodynamic efficiency of computations made in cells across the range of life
• Above and Beyond the Landauer Bound: Thermodynamics of Modularity
• Maximizing power and velocity of an information engine
• Biochemical machines for the interconversion of mutual information and work
• Landauer in the age of synthetic biology: energy consumption and information processing in biochemical networks
• Quantum Jarzynski Equality in Open Quantum Systems from the One-Time Measurement Scheme
• Fluctuation theorems from Bayesian retrodiction
• Thermodynamic limits to information harvesting by sensory systems
• Statistical Physics of Adaptation
• An energy-speed-accuracy relation in complex networks for biological discrimination
• Dependence of dissipation on the initial distribution over states
• Unified formalism for entropy productions and fluctuation relations
• Mutual Entropy-Production and Sensing in Bipartite Systems
• Global passivity in microscopic thermodynamics
• Energy and information flows in autonomous systems
• Thermodynamic costs of Turing Machines
• Information Geometric Inequalities of Chemical Thermodynamics
• Principles of Low Dissipation Computing from a Stochastic Circuit Model
• Thermodynamics of complexity and pattern manipulation
• Backward transfer entropy: Informational measure for detecting hidden Markov models and its interpretations in thermodynamics, gambling and causality
• Emergence of local irreversibility in complex interacting systems
• Unbounded memory advantage in stochastic simulation using quantum mechanics
• Marginal and Conditional Second Laws of Thermodynamics
• Tighter thermodynamic bound on speed limit in systems with unidirectional transitions
• Unified framework for the entropy production and the stochastic interaction based on information geometry
• Causal Asymmetry in a Quantum World
• The Free Energy Principle drives neuromorphic development
• Thermodynamic Neural Network
• The free energy requirements of biological organisms; implications for evolution
• Harnessing Fluctuations in Thermodynamic Computing via Time-Reversal Symmetries
• What we learn from the learning rate
• Bayesian Mechanics for Stationary Processes
• TherML: Thermodynamics of Machine Learning
• Optimized Bacteria are Environmental Prediction Engines
• Optimal high-level descriptions of dynamical systems
• The Emerging Energy Web
• Universal energy-accuracy tradeoffs in nonequilibrium cellular sensing
• Energetic cost of feedback control
• Thermodynamic Computing
• Fluctuations When Driving Between Nonequilibrium Steady States
• Trade-offs in delayed information transmission in biochemical networks
• Free-energy transduction within autonomous systems
• Nonequilibrium entropic bounds for Darwinian replicators
• Adaptive strategy in Kelly's horse races model
• Quantum Predictive Filtering
• Quantum correlations in predictive processes
• Additional energy-information relations in thermodynamics of small systems
• Entropy balance and Information processing in bipartite and non-bipartite composite systems
• Stochastic thermodynamics of interacting degrees of freedom: Fluctuation theorems for detached path probabilities
• Information Arbitrage in Bipartite Heat Engines
• Information thermodynamics: from physics to neuroscience
• Work statistics and thermal phase transitions
• Non-equilibrium odds for the emergence of life
• Predictive information in a nonequilibrium critical model
• Thermodynamics of Encoding and Encoders
• Provable superior accuracy in machine learned quantum models
• Relative cluster entropy for power-law correlated sequences
• Thermodynamic role of main reaction pathway and multi-body information flow in membrane transport
• On Improving the Performance of Nonphotochemical Quenching in CP29 Light-Harvesting Antenna Complex
• Thermodynamic fluctuation theorems govern human sensorimotor learning
• Learning in Neural Networks Based on a Generalized Fluctuation Theorem
• Kolmogorov-Sinai entropy and dissipation in driven classical Hamiltonian systems
• Contact geometry and quantum thermodynamics of nanoscale steady states
• Foundations of Stochastic Thermodynamics
• Energy dissipation and information flow in coupled Markovian systems
• Nonequilibrium work energy relation for non-Hamiltonian dynamics
• The second law and beyond in microscopic quantum setups
• Noise equals control
• Jarzyski's equality and Crooks' fluctuation theorem for general Markov chains with application to decision-making systems
• Environmental statistics and optimal regulation
• Dissipation in non-steady state regulatory circuits
• Information geometric complexity of entropic motion on curved statistical manifolds
• Bounds on the rates of statistical divergences and mutual information via stochastic thermodynamics
• The fundamental thermodynamic bounds on finite models
• On the thermodynamics of prediction under dissipative adaptation
• Bounded Rational Decision-Making in Changing Environments
• A framework for the use of generative modelling in non-equilibrium statistical mechanics
• Stochastic Thermodynamics of Learning Parametric Probabilistic Models
• Functional Decomposition and Estimation of Irreversibility in Time Series via Machine Learning
• Quantitative Nonequilibrium Pathway from Fundamental Physics to the Emergence and Persistence of Exoplanetary Biospheres
• A Non-equilibrium Thermodynamic Framework of Consciousness
• Input anticipating critical reservoirs show power law forgetting of unexpected input events
• Thermodynamic Limits of Physical Intelligence
• Memory shapes time perception and intertemporal choices