On the symmetrical Kullback-Leibler Jeffreys centroids

Abstract

Due to the success of the bag-of-word modeling paradigm, clustering histograms has become an important ingredient of modern information processing. Clustering histograms can be performed using the celebrated $k$-means centroid-based algorithm. From the viewpoint of applications, it is usually required to deal with symmetric distances. In this letter, we consider the Jeffreys divergence that symmetrizes the Kullback-Leibler divergence, and investigate the computation of Jeffreys centroids. We first prove that the Jeffreys centroid can be expressed analytically using the Lambert $W$ function for positive histograms. We then show how to obtain a fast guaranteed approximation when dealing with frequency histograms. Finally, we conclude with some remarks on the $k$-means histogram clustering.

17 pages, 1 figure, source code in R

References in corpus (2)

• The Burbea-Rao and Bhattacharyya centroids
• A family of statistical symmetric divergences based on Jensen's inequality

Cited by in corpus (11)

• Fast Computation of Wasserstein Barycenters
• On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
• On a generalization of the Jensen-Shannon divergence
• Semi-dual Regularized Optimal Transport
• Fast approximations of the Jeffreys divergence between univariate Gaussian mixture models via exponential polynomial densities
• A Smoothed Dual Approach for Variational Wasserstein Problems
• Proximity Operators of Discrete Information Divergences
• $\alpha$-Geodesical Skew Divergence
• New bounds for $k$-means and information $k$-means
• Learning from both experts and data
• Fast proxy centers for Jeffreys centroids: The Jeffreys-Fisher-Rao and the inductive Gauss-Bregman centers