A lower bound on the differential entropy of log-concave random vectors with applications

Abstract

We derive a lower bound on the differential entropy of a log-concave random variable $X$ in terms of the $p$-th absolute moment of $X$. The new bound leads to a reverse entropy power inequality with an explicit constant, and to new bounds on the rate-distortion function and the channel capacity. Specifically, we study the rate-distortion function for log-concave sources and distortion measure $|x-\hat{x}{|}^r$, and we establish that the difference between the rate distortion function and the Shannon lower bound is at most $\log (\sqrt{\pi e})\approx 1.5$ bits, independently of $r$ and the target distortion $d$. For mean-square error distortion, the difference is at most $\log (\sqrt{\frac{\pi e}{2}})\approx 1$ bits, regardless of $d$. We also provide bounds on the capacity of memoryless additive noise channels when the noise is log-concave. We show that the difference between the capacity of such channels and the capacity of the Gaussian channel with the same noise power is at most $\log (\sqrt{\frac{\pi e}{2}})\approx 1$ bits. Our results generalize to the case of vector $X$ with possibly dependent coordinates, and to $\gamma$-concave random variables. Our proof technique leverages tools from convex geometry.

31 pages, 2 figures

References in corpus (4)

• Forward and Reverse Entropy Power Inequalities in Convex Geometry
• A concavity property for the reciprocal of Fisher information and its consequences on Costa's EPI
• Wasserstein Stability of the Entropy Power Inequality for Log-Concave Densities
• Links between the Logarithmic Sobolev Inequality and the convolution inequalities for Entropy and Fisher Information

Cited by in corpus (13)

• Sharp moment-entropy inequalities and capacity bounds for log-concave distributions
• Pufferfish Privacy: An Information-Theoretic Study
• Sliced Mutual Information: A Scalable Measure of Statistical Dependence
• An Integral Representation of the Logarithmic Function with Applications in Information Theory
• Some Useful Integral Representations for Information-Theoretic Analyses
• Reversals of Rényi Entropy Inequalities under Log-Concavity
• Charge fluctuation entropy of Hawking radiation: a replica-free way to find large entropy
• Entropy-variance inequalities for discrete log-concave random variables via degree of freedom
• Differential Entropy of the Conditional Expectation under Additive Gaussian Noise
• Active embedding search via noisy paired comparisons
• Causality and Generalizability: Identifiability and Learning Methods
• Effective synchronization amid noise-induced chaos
• Support Recovery in the Phase Retrieval Model: Information-Theoretic Fundamental Limits