Monotonicity, thinning and discrete versions of the Entropy Power Inequality

Abstract

We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for Poisson variables. We show that some natural analogues of the Entropy Power Inequality do not in fact hold, but propose an alternative formulation which does always hold. The key to many proofs of Shannon's Entropy Power Inequality is the behaviour of entropy on scaling of continuous random variables. We believe that Rényi's operation of thinning discrete random variables plays a similar role to scaling, and give a sharp bound on how the entropy of ultra log-concave random variables behaves on thinning. In the spirit of the monotonicity results established by Artstein, Ball, Barthe and Naor, we prove a stronger version of concavity of entropy, which implies a strengthened form of our discrete Entropy Power Inequality.

9 pages (revised to take account of referees' comments)

References in corpus (2)

• Thinning, Entropy and the Law of Thin Numbers
• Concavity of entropy under thinning

Cited by in corpus (21)

• Note on Noisy Group Testing: Asymptotic Bounds and Belief Propagation Reconstruction
• Entropy bounds on abelian groups and the Ruzsa divergence
• Passive states optimize the output of bosonic Gaussian quantum channels
• Gaussian States Minimize the Output Entropy of the One-Mode Quantum Attenuator
• Entropy power inequalities for qudits
• Gaussian optimizers for entropic inequalities in quantum information
• Entropy Inequalities for Sums in Prime Cyclic Groups
• Majorization and Rényi Entropy Inequalities via Sperner Theory
• Entropy power inequality for a family of discrete random variables
• An Information-Theoretic Perspective of the Poisson Approximation via the Chen-Stein Method
• The One-Mode Quantum-Limited Gaussian Attenuator and Amplifier Have Gaussian Maximizers
• Bernoulli sums and Rényi entropy inequalities
• Thinning, photonic beamsplitting, and a general discrete entropy power inequality
• Entropies of weighted sums in cyclic groups and an application to polar codes
• On the Entropy of Sums of Bernoulli Random Variables via the Chen-Stein Method
• Continuity of mutual entropy in the large signal-to-noise ratio limit
• A de Bruijn identity for discrete random variables
• Adaptive sensing using deterministic partial Hadamard matrices
• Entropy and thinning of discrete random variables
• An Entropy Power Inequality for Discrete Random Variables
• A new entropy power inequality for integer-valued random variables