On the Kullback-Leibler divergence between discrete normal distributions
arXiv:2109.14920 · doi:10.1007/s41745-021-00279-5
Abstract
Discrete normal distributions are defined as the distributions with prescribed means and covariance matrices which maximize entropy on the integer lattice support. The set of discrete normal distributions form an exponential family with cumulant function related to the Riemann theta function. In this paper, we present several formula for common statistical divergences between discrete normal distributions including the Kullback-Leibler divergence. In particular, we describe an efficient approximation technique for calculating the Kullback-Leibler divergence between discrete normal distributions via the Rényi -divergences or the projective -divergences.
arXiv comment26 pages
References in corpus (8)
- Rényi Divergence and Kullback-Leibler Divergence
- An elementary introduction to information geometry
- The Burbea-Rao and Bhattacharyya centroids
- Bregman Voronoi Diagrams: Properties, Algorithms and Applications
- Chernoff information of exponential families
- On Hölder projective divergences
- Discrete Gaussian distributions via theta functions
- Computing Theta Functions with Julia