On Pinsker's Type Inequalities and Csiszar's f-divergences. Part I: Second and Fourth-Order Inequalities

Abstract

We study conditions on $f$ under which an $f$-divergence $D_f$ will satisfy $D_f\ge c_f{V}^2$ or $D_f\ge c_{2,f}{V}^2+c_{4,f}{V}^4$, where $V$ denotes variational distance and the coefficients $c_f$, $c_{2,f}$ and $c_{4,f}$ are {\em best possible}. As a consequence, we obtain lower bounds in terms of $V$ for many well known distance and divergence measures. For instance, let $D_{(\alpha )}(P,Q)=[\alpha (\alpha -1){]}^{-1}[\int q^{\alpha }{p}^{1-\alpha }d\mu -1]$ and ${\mathcal{I}}_{\alpha }(P,Q)=(\alpha -1{)}^{-1}\log [\int p^{\alpha }{q}^{1-\alpha }d\mu ]$ be respectively the {\em relative information of type} ($1-\alpha$) and {\em Rényi's information gain of order} $\alpha$. We show that $D_{(\alpha )}\ge 1/2V^2+1/72(\alpha +1)(2-\alpha )V^4$ whenever $-1\le \alpha \le 2$, $\alpha \ne 0,1$ and that ${\mathcal{I}}_{\alpha }=\frac{\alpha }{2}{V}^2+1/36\alpha (1+5\alpha -5{\alpha }^2)V^4$ for $0<\alpha <1$. Pinsker's inequality $D\ge 1/2V^2$ and its extension $D\ge 1/2V^2+1/36V^4$ are special cases of each one of these.

15 pages

Cited by in corpus (31)

• Rényi Divergence and Kullback-Leibler Divergence
• $f$-divergence Inequalities
• Different quantum f-divergences and the reversibility of quantum operations
• On $f$-Divergences: Integral Representations, Local Behavior, and Inequalities
• Theoretical evidence for adversarial robustness through randomization
• The Renyi Capacity and Center
• Scalable Information Inequalities for Uncertainty Quantification
• Computable Rényi mutual information: Area laws and correlations
• The Augustin Capacity and Center
• Sandwiched Rényi Convergence for Quantum Evolutions
• On Data-Processing and Majorization Inequalities for $f$-Divergences with Applications
• Internal entanglement and external correlations of any form limit each other
• Exponential decay of Rényi divergence under Fokker-Planck equations
• Bounds of the Pinsker and Fannes Types on the Tsallis Relative Entropy
• Amplifying Inter-message Distance: On Information Divergence Measures in Big Data
• Tight Bounds for Symmetric Divergence Measures and a New Inequality Relating $f$-Divergences
• Linear Bounds between Contraction Coefficients for $f$-Divergences
• On Reverse Pinsker Inequalities
• Tight Bounds for Symmetric Divergence Measures and a Refined Bound for Lossless Source Coding
• On some properties of Tsallis hypoentropies and hypodivergences
• Relating relative Rényi entropies and Wigner-Yanase-Dyson skew information to generalized multiple quantum coherences
• Properties of a Generalized Divergence Related to Tsallis Relative Entropy
• Rényi divergence and the central limit theorem
• Locally-Measured Rényi Divergences
• On quantum conditional entropies defined in terms of the $f$-divergences
• The Sphere Packing Bound for DSPCs with Feedback a la Augustin
• An Information-theoretic Approach to Distribution Shifts
• Erratum: Coding map for a contractive Markov system
• Dual Averaging on Compactly-Supported Distributions And Application to No-Regret Learning on a Continuum
• Minimum KL-divergence on complements of $L_1$ balls
• The Fourier Discrepancy Function