On a generalization of the Jensen-Shannon divergence

Abstract

The Jensen-Shannon divergence is a renown bounded symmetrization of the Kullback-Leibler divergence which does not require probability densities to have matching supports. In this paper, we introduce a vector-skew generalization of the scalar $\alpha$-Jensen-Bregman divergences and derive thereof the vector-skew $\alpha$-Jensen-Shannon divergences. We study the properties of these novel divergences and show how to build parametric families of symmetric Jensen-Shannon-type divergences. Finally, we report an iterative algorithm to numerically compute the Jensen-Shannon-type centroids for a set of probability densities belonging to a mixture family: This includes the case of the Jensen-Shannon centroid of a set of categorical distributions or normalized histograms.

19 pages, 3 figures

References in corpus (2)

• Information Entropy, Information Distances and Complexity in Atoms
• On Data-Processing and Majorization Inequalities for $f$-Divergences with Applications

Cited by in corpus (16)

• A review of Generative Adversarial Networks for Electronic Health Records: applications, evaluation measures and data sources
• Permutation Jensen-Shannon distance: A versatile and fast symbolic tool for complex time series analysis
• On a Variational Definition for the Jensen-Shannon Symmetrization of Distances based on the Information Radius
• On Relations Between the Relative entropy and ${\chi }^2$-Divergence, Generalizations and Applications
• Nonequilibrium chemical short-range order in metallic alloys
• Enhancing Fruit and Vegetable Detection in Unconstrained Environment with a Novel Dataset
• Heavy-tailed likelihoods for robustness against data outliers: Applications to the analysis of gravitational wave data
• Conditions for the existence of a generalization of Rényi divergence
• Information Interaction Profile of Choice Adoption
• Strongly Convex Divergences
• Transfer learning of state-based potential games for process optimization in decentralized manufacturing systems
• Systematic approaches to generate reversiblizations of Markov chains
• $\alpha$-Geodesical Skew Divergence
• Quantum speed limits based on Jensen-Shannon and Jeffreys divergences for general physical processes
• Fast proxy centers for Jeffreys centroids: The Jeffreys-Fisher-Rao and the inductive Gauss-Bregman centers
• Quantitative and Predictive Folding Models from Limited Single-Molecule Data Using Simulation-Based Inference