On Relations Between the Relative entropy and ${\chi }^2$-Divergence, Generalizations and Applications

Abstract

The relative entropy and chi-squared divergence are fundamental divergence measures in information theory and statistics. This paper is focused on a study of integral relations between the two divergences, the implications of these relations, their information-theoretic applications, and some generalizations pertaining to the rich class of $f$-divergences. Applications that are studied in this paper refer to lossless compression, the method of types and large deviations, strong~data-processing inequalities, bounds on contraction coefficients and maximal correlation, and the convergence rate to stationarity of a type of discrete-time Markov chains.

arXiv commentPublished in the Entropy journal, May 18, 2020. Journal version (open access) is available at https://www.mdpi.com/1099-4300/22/5/563

References in corpus (13)

• Rényi Divergence and Kullback-Leibler Divergence
• Sinc approximation of algebraically decaying functions
• $f$-divergence Inequalities
• On a generalization of the Jensen-Shannon divergence
• Beyond the entropy power inequality, via rearrangements
• Quantum Skew Divergence
• On $f$-Divergences: Integral Representations, Local Behavior, and Inequalities
• Comparison of Channels: Criteria for Domination by a Symmetric Channel
• On Data-Processing and Majorization Inequalities for $f$-Divergences with Applications
• The Differential Entropy of Mixtures: New Bounds and Applications
• Concentration of Measure Inequalities in Information Theory, Communications and Coding (Second Edition)
• Some Useful Integral Representations for Information-Theoretic Analyses
• A New Lower Bound for Kullback-Leibler Divergence Based on Hammersley-Chapman-Robbins Bound

Cited by in corpus (9)

• Unified Approach to Classical Speed Limit and Thermodynamic Uncertainty Relation
• On contraction coefficients, partial orders and approximation of capacities for quantum channels
• Irreversibility, Loschmidt echo, and thermodynamic uncertainty relation
• The Differential Entropy of Mixtures: New Bounds and Applications
• Thermodynamic uncertainty relation from involutions
• A Tight Lower Bound for the Hellinger Distance with Given Means and Variances
• Strongly Convex Divergences
• Relative Entropy as a Measure of Difference Between Hermitian and Non-Hermitian Systems
• Tight Lower Bounds for $\alpha$-Divergences Under Moment Constraints and Relations Between Different $\alpha$