Information geometry and sufficient statistics

Abstract

Information geometry provides a geometric approach to families of statistical models. The key geometric structures are the Fisher quadratic form and the Amari-Chentsov tensor. In statistics, the notion of sufficient statistic expresses the criterion for passing from one model to another without loss of information. This leads to the question how the geometric structures behave under such sufficient statistics. While this is well studied in the finite sample size case, in the infinite case, we encounter technical problems concerning the appropriate topologies. Here, we introduce notions of parametrized measure models and tensor fields on them that exhibit the right behavior under statistical transformations. Within this framework, we can then handle the topological issues and show that the Fisher metric and the Amari-Chentsov tensor on statistical models in the class of symmetric 2-tensor fields and 3-tensor fields can be uniquely (up to a constant) characterized by their invariance under sufficient statistics, thereby achieving a full generalization of the original result of Chentsov to infinite sample sizes. More generally, we decompose Markov morphisms between statistical models in terms of statistics. In particular, a monotonicity result for the Fisher information naturally follows.

37 p, final version, minor corrections, improved presentation

Cited by in corpus (23)

• Quantum information geometry of driven CFTs
• Information Geometry Formalism for the Spatially Homogeneous Boltzmann Equation
• On Closed-Form Expressions for the Fisher-Rao Distance
• From the Jordan product to Riemannian geometries on classical and quantum states
• Semi-invariant Riemannian metrics in hydrodynamics
• Fisher-Rao geometry of Dirichlet distributions
• A generalization of the maximum entropy principle for curved statistical manifolds
• Manifolds of classical probability distributions and quantum density operators in infinite dimensions
• On the Kähler Geometry of Certain Optimal Transport Problems
• Probabilistic morphisms and Bayesian nonparametrics
• Parametric models and information geometry on W*-algebras
• The Fisher-Rao Loss for Learning under Label Noise
• The $L^p$-Fisher-Rao metric and Amari-Cencov $\alpha$-connections
• Diffeological statistical models, the Fisher metric and probabilistic mappings
• The Markowitz Category
• The Design of Global Correlation Quantifiers and Continuous Notions of Statistical Sufficiency
• Geometry of the Fisher-Rao metric on the space of smooth densities on a compact manifold
• On Geodesic Completeness for Riemannian Metrics on Smooth Probability Densities
• Toward a relative q-entropy
• Geometric Structures Induced by Deformations of the Legendre Transform
• Natural differentiable structures on statistical models and the Fisher metric
• A Class of Non-Parametric Statistical Manifolds modelled on Sobolev Space
• Nonparametric estimations and the diffeological Fisher metric