An Entropy Power Inequality for Discrete Random Variables

Abstract

Let ${\mathsf{N}}_{\mathrm{d}}\left[X\right]=\frac{1}{2\pi \mathrm{e}}{\mathrm{e}}^{2\mathsf{H}\left[X\right]}$ denote the entropy power of the discrete random variable $X$ where $\mathsf{H}\left[X\right]$ denotes the discrete entropy of $X$. In this paper, we show that for two independent discrete random variables $X$ and $Y$, the entropy power inequality ${\mathsf{N}}_{\mathrm{d}}\left[X\right]+{\mathsf{N}}_{\mathrm{d}}\left[Y\right]\le 2{\mathsf{N}}_{\mathrm{d}}\left[X+Y\right]$ holds and it can be tight. The basic idea behind the proof is to perturb the discrete random variables using suitably designed continuous random variables. Then, the continuous entropy power inequality is applied to the sum of the perturbed random variables and the resulting lower bound is optimized.