The Burbea-Rao and Bhattacharyya centroids

Abstract

We study the centroid with respect to the class of information-theoretic Burbea-Rao divergences that generalize the celebrated Jensen-Shannon divergence by measuring the non-negative Jensen difference induced by a strictly convex and differentiable function. Although those Burbea-Rao divergences are symmetric by construction, they are not metric since they fail to satisfy the triangle inequality. We first explain how a particular symmetrization of Bregman divergences called Jensen-Bregman distances yields exactly those Burbea-Rao divergences. We then proceed by defining skew Burbea-Rao divergences, and show that skew Burbea-Rao divergences amount in limit cases to compute Bregman divergences. We then prove that Burbea-Rao centroids are unique, and can be arbitrarily finely approximated by a generic iterative concave-convex optimization algorithm with guaranteed convergence property. In the second part of the paper, we consider the Bhattacharyya distance that is commonly used to measure overlapping degree of probability distributions. We show that Bhattacharyya distances on members of the same statistical exponential family amount to calculate a Burbea-Rao divergence in disguise. Thus we get an efficient algorithm for computing the Bhattacharyya centroid of a set of parametric distributions belonging to the same exponential families, improving over former specialized methods found in the literature that were limited to univariate or "diagonal" multivariate Gaussians. To illustrate the performance of our Bhattacharyya/Burbea-Rao centroid algorithm, we present experimental performance results for $k$-means and hierarchical clustering methods of Gaussian mixture models.

13 pages

References in corpus (1)

• Statistical exponential families: A digest with flash cards

Cited by in corpus (41)

• On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
• An elementary introduction to information geometry
• On a generalization of the Jensen-Shannon divergence
• On the Chi square and higher-order Chi distances for approximating f-divergences
• Chernoff information of exponential families
• A family of statistical symmetric divergences based on Jensen's inequality
• Guaranteed bounds on the Kullback-Leibler divergence of univariate mixtures using piecewise log-sum-exp inequalities
• Matrix versions of the Hellinger distance
• Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means
• Total Jensen divergences: Definition, Properties and k-Means++ Clustering
• On a Variational Definition for the Jensen-Shannon Symmetrization of Distances based on the Information Radius
• On the symmetrical Kullback-Leibler Jeffreys centroids
• On the Kullback-Leibler divergence between discrete normal distributions
• Selective Filtering of Photonic Quantum Entanglement via Anti-Parity-Time Symmetry
• On Hölder projective divergences
• Interactive Music Generation with Positional Constraints using Anticipation-RNNs
• The duo Bregman and Fenchel-Young divergences
• Revisiting Chernoff Information with Likelihood Ratio Exponential Families
• Monte Carlo Information Geometry: The dually flat case
• On $w$-mixtures: Finite convex combinations of prescribed component distributions
• The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold
• On $f$-divergences between Cauchy distributions
• A note on Onicescu's informational energy and correlation coefficient in exponential families
• Cumulant-free closed-form formulas for some common (dis)similarities between densities of an exponential family
• Constructing the Matrix Multilayer Perceptron and its Application to the VAE
• A note on the $f$-divergences between multivariate location-scale families with either prescribed scale matrices or location parameters
• A generalization of the Jensen divergence: The chord gap divergence
• Systematic approaches to generate reversiblizations of Markov chains
• Generalizing Jensen and Bregman divergences with comparative convexity and the statistical Bhattacharyya distances with comparable means
• The α-divergences associated with a pair of strictly comparable quasi-arithmetic means
• Divergences induced by dual subtractive and divisive normalizations of exponential families and their convex deformations
• Bregman divergences based on optimal design criteria and simplicial measures of dispersion
• Variational Representations of Annealing Paths: Bregman Information under Monotonic Embedding
• Fast proxy centers for Jeffreys centroids: The Jeffreys-Fisher-Rao and the inductive Gauss-Bregman centers
• Approximate Bregman near neighbors in sublinear time: Beyond the triangle inequality
• The Bregman chord divergence
• On Voronoi diagrams and dual Delaunay complexes on the information-geometric Cauchy manifolds
• A note on some information-theoretic divergences between Zeta distributions
• Limit Theorems for Empirical Rényi Entropy and Divergence with Applications to Molecular Diversity Analysis
• Level Sets Based Distances for Probability Measures and Ensembles with Applications
• A note on the quasiconvex Jensen divergences and the quasiconvex Bregman divergences derived thereof