Thinning, Entropy and the Law of Thin Numbers

Abstract

Renyi's "thinning" operation on a discrete random variable is a natural discrete analog of the scaling operation for continuous random variables. The properties of thinning are investigated in an information-theoretic context, especially in connection with information-theoretic inequalities related to Poisson approximation results. The classical Binomial-to-Poisson convergence (sometimes referred to as the "law of small numbers" is seen to be a special case of a thinning limit theorem for convolutions of discrete distributions. A rate of convergence is provided for this limit, and nonasymptotic bounds are also established. This development parallels, in part, the development of Gaussian inequalities leading to the information-theoretic version of the central limit theorem. In particular, a "thinning Markov chain" is introduced, and it is shown to play a role analogous to that of the Ornstein-Uhlenbeck process in connection to the entropy power inequality.

References in corpus (1)

• Sharp Bounds on the Entropy of the Poisson Law and Related Quantities

Cited by in corpus (20)

• Monotonicity, thinning and discrete versions of the Entropy Power Inequality
• Passive states optimize the output of bosonic Gaussian quantum channels
• Gaussian States Minimize the Output Entropy of the One-Mode Quantum Attenuator
• Gaussian optimizers for entropic inequalities in quantum information
• Entropy Inequalities for Sums in Prime Cyclic Groups
• An Information-Theoretic Perspective of the Poisson Approximation via the Chen-Stein Method
• The One-Mode Quantum-Limited Gaussian Attenuator and Amplifier Have Gaussian Maximizers
• Entropy power inequality for a family of discrete random variables
• Throwing Stones and Collecting Bones: Looking for Poisson-like Random Measures
• Bernoulli sums and Rényi entropy inequalities
• Entropies of weighted sums in cyclic groups and an application to polar codes
• On the Entropy of Sums of Bernoulli Random Variables via the Chen-Stein Method
• Thinning and Information Projections
• A de Bruijn identity for discrete random variables
• Rate Distortion Theory for Descriptive Statistics
• The Degraded Discrete-Time Poisson Wiretap Channel
• Entropy and thinning of discrete random variables
• A negative binomial approximation in group testing
• The law of thin processes: a law of large numbers for point processes
• Discrete Dispersion Models and Their Tweedie Asymptotics