On Rényi and Tsallis entropies and divergences for exponential families

Abstract

Many common probability distributions in statistics like the Gaussian, multinomial, Beta or Gamma distributions can be studied under the unified framework of exponential families. In this paper, we prove that both Rényi and Tsallis divergences of distributions belonging to the same exponential family admit a generic closed form expression. Furthermore, we show that Rényi and Tsallis entropies can also be calculated in closed-form for sub-families including the Gaussian or exponential distributions, among others.

7 pages

References in corpus (2)

• A class of Rényi information estimators for multidimensional densities
• Note on exponential families of distributions

Cited by in corpus (17)

• On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means
• Guaranteed bounds on the Kullback-Leibler divergence of univariate mixtures using piecewise log-sum-exp inequalities
• Rényi entropy and complexity measure for skew-gaussian distributions and related families
• On Hölder projective divergences
• q-VAE for Disentangled Representation Learning and Latent Dynamical Systems
• Exponential decay of Rényi divergence under Fokker-Planck equations
• Rényi entropies for multidimensional hydrogenic systems in position and momentum spaces
• A series of maximum entropy upper bounds of the differential entropy
• A new class of entropic information measures, formal group theory and information geometry
• Sharma-Mittal Quantum Discord
• A note on Onicescu's informational energy and correlation coefficient in exponential families
• Quantum Mechanics Based on an Extended Least Action Principle and Information Metrics of Vacuum Fluctuations
• Quantum Scalar Field Theory Based on the Extended Least Action Principle
• Information divergences and likelihood ratios of Poisson processes and point patterns
• Spin Theory Based on the Extended Least Action Principle and Information Metrics: Quantization, Entanglement, and Bell Test With Time Delay
• Entanglement Trajectory and its Boundary
• Alternative Framework to Quantize Fermionic Fields