A generalization of the maximum entropy principle for curved statistical manifolds
arXiv:2105.07953 · doi:10.1103/PhysRevResearch.3.033216
Abstract
The maximum entropy principle (MEP) is one of the most prominent methods to investigate and model complex systems. Despite its popularity, the standard form of the MEP can only generate Boltzmann-Gibbs distributions, which are ill-suited for many scenarios of interest. As a principled approach to extend the reach of the MEP, this paper revisits its foundations in information geometry and shows how the geometry of curved statistical manifolds naturally leads to a generalization of the MEP based on the Rényi entropy. By establishing a bridge between non-Euclidean geometry and the MEP, our proposal sets a solid foundation for the numerous applications of the Rényi entropy, and enables a range of novel methods for complex systems analysis.
References in corpus (36)
- Weak pairwise correlations imply strongly correlated network states in a neural population
- Direct-coupling analysis of residue co-evolution captures native contacts across many protein families
- Measuring entanglement entropy through the interference of quantum many-body twins
- Identification of direct residue contacts in protein-protein interaction by message passing
- Superstatistics
- Statistical mechanics for natural flocks of birds
- Measuring Renyi Entanglement Entropy with Quantum Monte Carlo
- Maximum entropy models for antibody diversity
- The Gravity Dual of Renyi Entropy
- Network information and connected correlations
- Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information
- Asymptotically scale-invariant occupancy of phase space makes the entropy Sq extensive
- Holographic entanglement beyond classical gravity
- Quantifying selection in immune receptor repertoires
- Partial Information Decomposition as a Unified Approach to the Specification of Neural Goal Functions
- Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data
- Prediction of spatio-temporal patterns of neural activity from pairwise correlations
- The dynamics of metric-affine gravity
- Maximum Entropy Principle in statistical inference: case for non-Shannonian entropies
- Logarithmic terms in entanglement entropies of 2D quantum critical points and Shannon entropies of spin chains
- Dynamical maximum entropy approach to flocking
- Information geometry and sufficient statistics
- Shannon and Rényi mutual information in quantum critical spin chains
- On observability of Renyi's entropy
- The role of nonmetricity in metric-affine theories of gravity
- Detecting Classical Phase Transitions with Renyi Mutual Information
- An information-theoretic approach to self-organisation: Emergence of complex interdependencies in coupled dynamical systems
- Understanding interdependency through complex information sharing
- Beyond position weight matrices: nucleotide correlations in transcription factor binding sites and their description
- Generalized entropies and logarithms and their duality relations
- Exact computation of the Maximum Entropy Potential of spiking neural networks models
- Geometric mutual information at classical critical points
- Group entropies: from phase space geometry to entropy functionals via group theory
- Surges of collective human activity emerge from simple pairwise correlations
- The world according to Renyi: Thermodynamics of multifractal systems
- Kählerian information geometry for signal processing