An elementary introduction to information geometry

Abstract

In this survey, we describe the fundamental differential-geometric structures of information manifolds, state the fundamental theorem of information geometry, and illustrate some use cases of these information manifolds in information sciences. The exposition is self-contained by concisely introducing the necessary concepts of differential geometry, but proofs are omitted for brevity.

56 pages, 16 figures

References in corpus (12)

• Geometrothermodynamics
• On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
• Ruppeiner Geometry, Phase Transitions, and the Microstructure of Charged AdS Black Holes
• Relative Loss Bounds for On-line Density Estimation with the Exponential Family of Distributions
• Revisiting Chernoff Information with Likelihood Ratio Exponential Families
• Agglomerative Bregman Clustering
• Monte Carlo Information Geometry: The dually flat case
• Cumulant-free closed-form formulas for some common (dis)similarities between densities of an exponential family
• A Riemanian Approach to Blob Detection in Manifold-Valued Images
• On geodesic triangles with right angles in a dually flat space
• The Bregman chord divergence
• The Epic Story of Maximum Likelihood

Cited by in corpus (58)

• On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
• Target Detection within Nonhomogeneous Clutter via Total Bregman Divergence-Based Matrix Information Geometry Detectors
• Robust And Scalable Learning Of Complex Dataset Topologies Via Elpigraph
• Universal Bound on Energy Cost of Bit Reset in Finite Time
• Unsupervised Learning Discriminative MIG Detectors in Nonhomogeneous Clutter
• LDA-MIG Detectors for Maritime Targets in Nonhomogeneous Sea Clutter
• On the Kullback-Leibler divergence between discrete normal distributions
• The duo Bregman and Fenchel-Young divergences
• On Closed-Form Expressions for the Fisher-Rao Distance
• Bayesian Renormalization
• Intrinsic Quantum Mpemba Effect in Markovian Systems and Quantum Circuits
• Information geometry approach to quantum stochastic thermodynamics
• Fast approximations of the Jeffreys divergence between univariate Gaussian mixture models via exponential polynomial densities
• Hilbert geometry of the Siegel disk: The Siegel-Klein disk model
• Length-preserving biconnection gravity and its cosmological implications
• On $f$-divergences between Cauchy distributions
• Proximal Galerkin: A structure-preserving finite element method for pointwise bound constraints
• Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents
• Information Thermodynamics of the Transition-Path Ensemble
• Principal component analysis for Gaussian process posteriors
• The Hookean Law of Black Holes and Fragmentation: Insights from Maximum Force Conjecture and Ruppeiner Geometry
• Cumulant-free closed-form formulas for some common (dis)similarities between densities of an exponential family
• On the Importance of Firth Bias Reduction in Few-Shot Classification
• On information projections between multivariate elliptical and location-scale families
• Einstein's equations and the pseudo-entropy of pseudo-Riemannian information manifolds
• Annealed Importance Sampling with q-Paths
• Weyl Prior and Bayesian Statistics
• Entropic Dynamics of Networks
• Bayesian RG Flow in Neural Network Field Theories
• Abundant Superintegrable Systems and Hessian Structures
• Infinite Distance Limits and Information Theory
• Statistical Einstein manifolds of exponential families with group-invariant potential functions
• A Fisher information-based incompatibility criterion for quantum channels
• NCoder -- A Quantum Field Theory approach to encoding data
• The α-divergences associated with a pair of strictly comparable quasi-arithmetic means
• A Unifying and Canonical Description of Measure-Preserving Diffusions
• An information-geometric approach to feature extraction and moment reconstruction in dynamical systems
• Cartan moving frames and the data manifolds
• Information geometry and Bose-Einstein condensation
• On the criticality of the configuration-space statistical geometry
• Relations between entropy rate, entropy production and information geometry in linear stochastic systems
• Vertex functions and their flow equations from the 2PI effective action
• Variational Representations of Annealing Paths: Bregman Information under Monotonic Embedding
• Non-equilibrium thermodynamics with first-passage time of states as independent thermodynamic parameter
• Statistical Geometry and Hessian Structures on Pre-Leibniz Algebroids
• On Voronoi diagrams and dual Delaunay complexes on the information-geometric Cauchy manifolds
• Geodesic curves in Gaussian random field manifolds
• On the curvatures of Gaussian random field manifolds
• On the Variance of the Fisher Information for Deep Learning
• Quantifying Irreversibility via Bayesian Subjectivity for Classical & Quantum Linear Maps
• Information geometry and $\alpha$-parallel prior of the beta-logistic distribution
• Adversarial attacks on neural networks through canonical Riemannian foliations
• From generalized arithmetic means to geodesics to Hamilton dynamics to Bregman divergences
• Pulling back information geometry
• Entropic dynamics yields reciprocal relations
• Legendre transformation and information geometry for the maximum entropy theory of ecology
• Kullback-Leibler Divergence-Based Out-of-Distribution Detection with Flow-Based Generative Models
• Geometry and clustering with metrics derived from separable Bregman divergences