Monotonic Convergence in an Information-Theoretic Law of Small Numbers
arXiv:0810.5203 · doi:10.1109/TIT.2009.2032727
Abstract
An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic convergence in relative entropy is established for general discrete distributions, while monotonic increase of Shannon entropy is proved for the special class of ultra-log-concave distributions. Overall we extend the parallel between the information-theoretic central limit theorem and law of small numbers explored by Kontoyiannis et al. (2005) and Harremoës et al.\ (2007, 2008). Ingredients in the proofs include convexity, majorization, and stochastic orders.
minor changes; references added
References in corpus (5)
- Generalized Entropy Power Inequalities and Monotonicity Properties of Information
- Log-concavity and the maximum entropy property of the Poisson distribution
- On an Inequality of Karlin and Rinott Concerning Weighted Sums of i.i.d. Random Variables
- Concavity of entropy under thinning
- Relative log-concavity and a pair of triangle inequalities
Cited by in corpus (16)
- Forward and Reverse Entropy Power Inequalities in Convex Geometry
- Sharp Bounds on the Entropy of the Poisson Law and Related Quantities
- Thinning, Entropy and the Law of Thin Numbers
- Monotonicity, thinning and discrete versions of the Entropy Power Inequality
- Passive states optimize the output of bosonic Gaussian quantum channels
- Gaussian States Minimize the Output Entropy of the One-Mode Quantum Attenuator
- Gaussian optimizers for entropic inequalities in quantum information
- A discrete log-Sobolev inequality under a Bakry-Emery type condition
- Combinatorial Entropy Power Inequalities: A Preliminary Study of the Stam region
- Entropy Inequalities for Sums in Prime Cyclic Groups
- Concavity of entropy under thinning
- The One-Mode Quantum-Limited Gaussian Attenuator and Amplifier Have Gaussian Maximizers
- Discrete versions of the transport equation and the Shepp-Olkin conjecture
- Thinning, photonic beamsplitting, and a general discrete entropy power inequality
- On the Entropy of Sums of Bernoulli Random Variables via the Chen-Stein Method
- Entropy and thinning of discrete random variables