Concavity of entropy under thinning

Abstract

Building on the recent work of Johnson (2007) and Yu (2008), we prove that entropy is a concave function with respect to the thinning operation T_a. That is, if X and Y are independent random variables on Z_+ with ultra-log-concave probability mass functions, then H(T_a X+T_{1-a} Y)>= a H(X)+(1-a)H(Y), 0 <= a <= 1, where H denotes the discrete entropy. This is a discrete analogue of the inequality (h denotes the differential entropy) h(sqrt(a) X + sqrt{1-a} Y)>= a h(X)+(1-a) h(Y), 0 <= a <= 1, which holds for continuous X and Y with finite variances and is equivalent to Shannon's entropy power inequality. As a consequence we establish a special case of a conjecture of Shepp and Olkin (1981).

To be presented at ISIT09

References in corpus (3)

• Log-concavity and the maximum entropy property of the Poisson distribution
• Entropy and the Law of Small Numbers
• Monotonic Convergence in an Information-Theoretic Law of Small Numbers

Cited by in corpus (10)

• Information Theoretic Proofs of Entropy Power Inequalities
• Monotonicity, thinning and discrete versions of the Entropy Power Inequality
• Passive states optimize the output of bosonic Gaussian quantum channels
• Gaussian States Minimize the Output Entropy of the One-Mode Quantum Attenuator
• Monotonic Convergence in an Information-Theoretic Law of Small Numbers
• A proof of the Shepp-Olkin entropy concavity conjecture
• Discrete versions of the transport equation and the Shepp-Olkin conjecture
• Thinning, photonic beamsplitting, and a general discrete entropy power inequality
• A proof of the Shepp-Olkin entropy monotonicity conjecture
• Entropy and thinning of discrete random variables