Bounds on the rates of statistical divergences and mutual information via stochastic thermodynamics
arXiv:2308.05597 · doi:10.1103/PhysRevE.109.054126
Abstract
Statistical divergences are important tools in data analysis, information theory, and statistical physics, and there exist well known inequalities on their bounds. However, in many circumstances involving temporal evolution, one needs limitations on the rates of such quantities, instead. Here, several general upper bounds on the rates of some f-divergences are derived, valid for any type of stochastic dynamics (both Markovian and non-Markovian), in terms of information-like and/or thermodynamic observables. As special cases, the analytical bounds on the rate of mutual information are obtained. The major role in all those limitations is played by temporal Fisher information, characterizing the speed of global system dynamics, and some of them contain entropy production, suggesting a link with stochastic thermodynamics. Indeed, the derived inequalities can be used for estimation of minimal dissipation and global speed in thermodynamic stochastic systems. Specific applications of these inequalities in physics and neuroscience are given, which include the bounds on the rates of free energy and work in nonequilibrium systems, limits on the speed of information gain in learning synapses, as well as the bounds on the speed of predictive inference and learning rate. Overall, the derived bounds can be applied to any complex network of interacting elements, where predictability and thermodynamics of network dynamics are of prime concern.
arXiv commentReplacement: correction of typos in Eqs. (56), (57), and around them. Change of license to CC BY
References in corpus (39)
- Estimating Mutual Information
- Thermodynamic uncertainty relation for biomolecular processes
- The maximum speed of dynamical evolution
- Steady State Thermodynamics of Langevin Systems
- Quantum speed limits: from Heisenberg's uncertainty principle to optimal quantum control
- Measuring thermodynamic length
- Second law and Landauer principle far from equilibrium
- Fluctuations and response of nonequilibrium states
- The Three Faces of the Second Law: I. Master Equation Formulation
- -divergence Inequalities
- The thermodynamics of prediction
- Thermodynamics with continuous information flow
- Speed Limit for Classical Stochastic Processes
- Quantifying dissipation using fluctuating currents
- Stochastic thermodynamics of bipartite systems: transfer entropy inequalities and a Maxwell's demon interpretation
- Quantum Speed Limits Across the Quantum-to-Classical Transition
- Thermodynamic uncertainty relation for time-dependent driving
- Thermodynamic Unification of Optimal Transport: Thermodynamic Uncertainty Relation, Minimum Dissipation, and Thermodynamic Speed Limits
- Stochastic thermodynamics of computation
- Entropy and mutual information in models of deep neural networks
- Time-information uncertainty relations in thermodynamics
- Frenesy: time-symmetric dynamical activity in nonequilibria
- Entropy production estimation with optimal current
- Stochastic time-evolution, information geometry and the Cramer-Rao Bound
- Thermodynamics of statistical inference by cells
- Thermodynamic efficiency of information and heat flow
- Unifying Quantum and Classical Speed Limits on Observables
- Minimum entropy production, detailed balance and Wasserstein distance for continuous-time Markov processes
- Estimating Mixture Entropy with Pairwise Distances
- Information-theoretic vs. thermodynamic entropy production in autonomous sensory networks
- Unified thermodynamic-kinetic uncertainty relation
- Metabolic constraints on synaptic learning and memory
- Mutual Entropy-Production and Sensing in Bipartite Systems
- Energy and information flows in autonomous systems
- Beyond Thermodynamic Uncertainty Relations: nonlinear response, error-dissipation trade-offs, and speed limits
- Phase space gradient of dissipated work and information: A role of relative Fisher information
- General H-theorem and entropies that violate the second law
- de Bruijn-type identity for systems with flux
- Cooperativity, information gain, and energy cost during early LTP in dendritic spines