Chernoff information of exponential families

Abstract

Chernoff information upper bounds the probability of error of the optimal Bayesian decision rule for $2$-class classification problems. However, it turns out that in practice the Chernoff bound is hard to calculate or even approximate. In statistics, many usual distributions, such as Gaussians, Poissons or frequency histograms called multinomials, can be handled in the unified framework of exponential families. In this note, we prove that the Chernoff information for members of the same exponential family can be either derived analytically in closed form, or efficiently approximated using a simple geodesic bisection optimization technique based on an exact geometric characterization of the "Chernoff point" on the underlying statistical manifold.

References in corpus (3)

• The Burbea-Rao and Bhattacharyya centroids
• Statistical exponential families: A digest with flash cards
• Note on exponential families of distributions

Cited by in corpus (19)

• On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
• An elementary introduction to information geometry
• Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means
• Arithmetic Average Density Fusion -- Part I: Some Statistic and Information-theoretic Results
• On a Variational Definition for the Jensen-Shannon Symmetrization of Distances based on the Information Radius
• Multi-sensor Suboptimal Fusion Student's $t$ Filter
• On the Kullback-Leibler divergence between discrete normal distributions
• Revisiting Chernoff Information with Likelihood Ratio Exponential Families
• Monte Carlo Information Geometry: The dually flat case
• On $w$-mixtures: Finite convex combinations of prescribed component distributions
• On $f$-divergences between Cauchy distributions
• Quantum Target Ranging for LiDAR
• Quantum illumination with noisy probes: Conditional advantages of non-Gaussianity
• Cumulant-free closed-form formulas for some common (dis)similarities between densities of an exponential family
• Information divergences and likelihood ratios of Poisson processes and point patterns
• Likelihood Ratio Exponential Families
• Chernoff Information Bottleneck for Covert Quantum Target Sensing
• On Voronoi diagrams and dual Delaunay complexes on the information-geometric Cauchy manifolds
• Information Theoretic Bounds on Optimal Worst-case Error in Binary Mixture Identification