On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means

Abstract

The Jensen-Shannon divergence is a renown bounded symmetrization of the unbounded Kullback-Leibler divergence which measures the total Kullback-Leibler divergence to the average mixture distribution. However the Jensen-Shannon divergence between Gaussian distributions is not available in closed-form. To bypass this problem, we present a generalization of the Jensen-Shannon (JS) divergence using abstract means which yields closed-form expressions when the mean is chosen according to the parametric family of distributions. More generally, we define the JS-symmetrizations of any distance using generalized statistical mixtures derived from abstract means. In particular, we first show that the geometric mean is well-suited for exponential families, and report two closed-form formula for (i) the geometric Jensen-Shannon divergence between probability densities of the same exponential family, and (ii) the geometric JS-symmetrization of the reverse Kullback-Leibler divergence. As a second illustrating example, we show that the harmonic mean is well-suited for the scale Cauchy distributions, and report a closed-form formula for the harmonic Jensen-Shannon divergence between scale Cauchy distributions. We also define generalized Jensen-Shannon divergences between matrices (e.g., quantum Jensen-Shannon divergences) and consider clustering with respect to these novel Jensen-Shannon divergences.

33 pages

References in corpus (6)

• On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
• An elementary introduction to information geometry
• Properties of Classical and Quantum Jensen-Shannon Divergence
• Strong Convexity of Sandwiched Entropies and Related Optimization Problems
• Monte Carlo Information Geometry: The dually flat case
• Constraining Variational Inference with Geometric Jensen-Shannon Divergence

Cited by in corpus (43)

• On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
• An elementary introduction to information geometry
• On a generalization of the Jensen-Shannon divergence
• (H)DPGMM: A Hierarchy of Dirichlet Process Gaussian Mixture Models for the inference of the black hole mass function
• On a Variational Definition for the Jensen-Shannon Symmetrization of Distances based on the Information Radius
• Simulation of complex dynamics of mean-field $p$-spin models using measurement-based quantum feedback control
• Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels
• Jensen-Shannon Divergence Based Novel Loss Functions for Bayesian Neural Networks
• Concept Drift Adaptation in Text Stream Mining Settings: A Systematic Review
• Revisiting Chernoff Information with Likelihood Ratio Exponential Families
• Broken detailed balance and entropy production in directed networks
• A numerical approximation method for the Fisher-Rao distance between multivariate normal distributions
• Fast approximations of the Jeffreys divergence between univariate Gaussian mixture models via exponential polynomial densities
• Uncovering dark matter density profiles in dwarf galaxies with graph neural networks
• A closed-form formula for the Kullback-Leibler divergence between Cauchy distributions
• On $f$-divergences between Cauchy distributions
• Diversity Actor-Critic: Sample-Aware Entropy Regularization for Sample-Efficient Exploration
• Adaptive Kernel Density Estimation proposal in gravitational wave data analysis
• Reconstructing parametric gravitational-wave population fits from non-parametric results without refitting the data
• When Invariant Representation Learning Meets Label Shift: Insufficiency and Theoretical Insights
• Inferring Cosmological Parameters with Evidential Physics-Informed Neural Networks
• Solar Wind Structures from the Gaussianity of Magnetic Magnitude
• Impact of the observation frequency coverage on the significance of a gravitational wave background detection in PTA data
• Transmission distance in the space of quantum channels
• Agile Climate-Sensor Design and Calibration Algorithms Using Machine Learning: Experiments From Cape Point
• Modeling Eye Gaze Velocity Trajectories using GANs with Spectral Loss for Enhanced Fidelity
• Berezinskii-Kosterlitz-Thouless transition from Neural Network Flows
• Cumulant-free closed-form formulas for some common (dis)similarities between densities of an exponential family
• Information-Geometric Set Embeddings (IGSE): From Sets to Probability Distributions
• Transport information Bregman divergences
• Statistical complexity and the road to equilibrium in many-body chaotic quantum systems
• $\alpha$-Geodesical Skew Divergence
• CIM-PPO:Proximal Policy Optimization with Liu-Correntropy Induced Metric
• Through-Wall Imaging based on WiFi Channel State Information
• New bounds for $k$-means and information $k$-means
• Ubiquitous Dynamical Time Asymmetry in Measurements on Materials and Biological Systems
• A Complexity-Based Approach to Quantum Observable Equilibration
• Modeling stochastic eye tracking data: A comparison of quantum generative adversarial networks and Markov models
• q-Paths: Generalizing the Geometric Annealing Path using Power Means
• Fast proxy centers for Jeffreys centroids: The Jeffreys-Fisher-Rao and the inductive Gauss-Bregman centers
• A note on the quasiconvex Jensen divergences and the quasiconvex Bregman divergences derived thereof
• Estimating Free Parameters in Stochastic Oscillatory Models Using a Weighted Cost Function
• Orientation-Dependent Protein Binding at Nanoparticle Interfaces