Statistical exponential families: A digest with flash cards

Abstract

This document describes concisely the ubiquitous class of exponential family distributions met in statistics. The first part recalls definitions and summarizes main properties and duality with Bregman divergences (all proofs are skipped). The second part lists decompositions and related formula of common exponential family distributions. We recall the Fisher-Rao-Riemannian geometries and the dual affine connection information geometries of statistical manifolds. It is intended to maintain and update this document and catalog by adding new distribution items.

Cited by in corpus (38)

• On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
• An elementary introduction to information geometry
• The Burbea-Rao and Bhattacharyya centroids
• Chernoff information of exponential families
• Joint Color-Spatial-Directional clustering and Region Merging (JCSD-RM) for unsupervised RGB-D image segmentation
• $k$-MLE: A fast algorithm for learning statistical mixture models
• Information geometry for Fermi-Dirac and Bose-Einstein quantum statistics
• Entropic dynamics on Gibbs statistical manifolds
• Fast approximations of the Jeffreys divergence between univariate Gaussian mixture models via exponential polynomial densities
• Monte Carlo Information Geometry: The dually flat case
• A series of maximum entropy upper bounds of the differential entropy
• On $w$-mixtures: Finite convex combinations of prescribed component distributions
• Poisson noise reduction with non-local PCA
• Implementing a Bayes Filter in a Neural Circuit: The Case of Unknown Stimulus Dynamics
• Approximate Decentralized Bayesian Inference
• Log-Linear RNNs: Towards Recurrent Neural Networks with Flexible Prior Knowledge
• The Bayesian Learning Rule
• Advantages of $q$-logarithm representation over $q$-exponential representation from the sense of scale and shift on nonlinear systems
• A universal framework for learning the elliptical mixture model
• A note on Onicescu's informational energy and correlation coefficient in exponential families
• On information projections between multivariate elliptical and location-scale families
• Cumulant-free closed-form formulas for some common (dis)similarities between densities of an exponential family
• Variational Bayes Inference in Digital Receivers
• Stochastic thermodynamic limit on E. coli adaptation by Information geometric approach
• On the Kullback-Leibler divergence between location-scale densities
• Latent Variable Modeling for Generative Concept Representations and Deep Generative Models
• On power chi expansions of $f$-divergences
• Statistical Einstein manifolds of exponential families with group-invariant potential functions
• Faster Stochastic Variational Inference using Proximal-Gradient Methods with General Divergence Functions
• Optimal Learning of Mallows Block Model
• On The Chain Rule Optimal Transport Distance
• A Probabilistic Representation of Deep Learning
• Comparing Comparators in Generalization Bounds
• Stochastic Discriminative EM
• Explicitly Bayesian Regularizations in Deep Learning
• A Theoretical Analysis of Logistic Regression and Bayesian Classifiers
• A note on some information-theoretic divergences between Zeta distributions
• Model Inconsistent but Correlated Noise: Multi-view Subspace Learning with Regularized Mixture of Gaussians