The Shannon Lower Bound is Asymptotically Tight

Abstract

The Shannon lower bound is one of the few lower bounds on the rate-distortion function that holds for a large class of sources. In this paper, it is demonstrated that its gap to the rate-distortion function vanishes as the allowed distortion tends to zero for all sources having a finite differential entropy and whose integer part is finite. Conversely, it is demonstrated that if the integer part of the source has an infinite entropy, then its rate-distortion function is infinite for every finite distortion. Consequently, the Shannon lower bound provides an asymptotically tight bound on the rate-distortion function if, and only if, the integer part of the source has a finite entropy.

13 pages, no figures. Replaced with version that has been submitted to IEEE Transactions on Information Theory. Lemma 1 has been generalized

References in corpus (1)

• Degrees of freedom in vector interference channels

Cited by in corpus (8)

• A lower bound on the differential entropy of log-concave random vectors with applications
• On the Information Dimension of Stochastic Processes
• On the Information Dimension of Multivariate Gaussian Processes
• Rate-Distortion Theory of Finite Point Processes
• Rate-Distortion Analysis of Minimum Excess Risk in Bayesian Learning
• The CEO Problem with $r$th Power of Difference and Logarithmic Distortions
• Shannon Bounds for Quadratic Rate-Distortion Problems
• Converse Bounds for Entropy-Constrained Quantization Via a Variational Entropy Inequality