Revisiting Chernoff Information with Likelihood Ratio Exponential Families

Abstract

The Chernoff information between two probability measures is a statistical divergence measuring their deviation defined as their maximally skewed Bhattacharyya distance. Although the Chernoff information was originally introduced for bounding the Bayes error in statistical hypothesis testing, the divergence found many other applications due to its empirical robustness property found in applications ranging from information fusion to quantum information. From the viewpoint of information theory, the Chernoff information can also be interpreted as a minmax symmetrization of the Kullback--Leibler divergence. In this paper, we first revisit the Chernoff information between two densities of a measurable Lebesgue space by considering the exponential families induced by their geometric mixtures: The so-called likelihood ratio exponential families. Second, we show how to (i) solve exactly the Chernoff information between any two univariate Gaussian distributions or get a closed-form formula using symbolic computing, (ii) report a closed-form formula of the Chernoff information of centered Gaussians with scaled covariance matrices and (iii) use a fast numerical scheme to approximate the Chernoff information between any two multivariate Gaussian distributions.

arXiv comment41 pages

References in corpus (11)

• Rényi Divergence and Kullback-Leibler Divergence
• The Quantum Chernoff Bound
• Asymptotic Error Rates in Quantum Hypothesis Testing
• On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
• The Burbea-Rao and Bhattacharyya centroids
• Statistical inference on random dot product graphs: a survey
• Bregman Voronoi Diagrams: Properties, Algorithms and Applications
• Chernoff information of exponential families
• Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means
• Is There a Trade-Off Between Fairness and Accuracy? A Perspective Using Mismatched Hypothesis Testing
• On a Variational Definition for the Jensen-Shannon Symmetrization of Distances based on the Information Radius

Cited by in corpus (3)

• An elementary introduction to information geometry
• Quantum Target Ranging for LiDAR
• Fast proxy centers for Jeffreys centroids: The Jeffreys-Fisher-Rao and the inductive Gauss-Bregman centers