Entropy Bounds for Discrete Random Variables via Maximal Coupling
arXiv:1209.5259 · doi:10.1109/TIT.2013.2274515
Abstract
This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal coupling, and they apply to discrete random variables which are defined over finite or countably infinite alphabets. Loosened versions of these bounds are demonstrated to reproduce some previously reported results. The use of the new bounds is exemplified for the Poisson approximation, where bounds on the local and total variation distances follow from Stein's method.
Final version. Accepted to the IEEE Trans. on Information Theory, July 2013
References in corpus (4)
Cited by in corpus (13)
- -divergence Inequalities
- Tight uniform continuity bounds for quantum entropies: conditional entropy, relative entropy distance and energy constraints
- On the Entropy of Couplings
- From Classical to Quantum: Uniform Continuity Bounds on Entropies in Infinite Dimensions
- Efficient Approximate Minimum Entropy Coupling of Multiple Probability Distributions
- A tight uniform continuity bound for the Arimoto-Rényi conditional entropy and its extension to classical-quantum states
- Information Losses in Neural Classifiers from Sampling
- A Tight Uniform Continuity Bound for Equivocation
- Continuity of entropies via integral representations
- IT formulae for gamma target: mutual information and relative entropy
- Generalizations of Fano's Inequality for Conditional Information Measures via Majorization Theory
- Optimizing Linear Correctors: A Tight Output Min-Entropy Bound and Selection Technique
- Tightening continuity bounds for entropies and bounds on quantum capacities