Compound Poisson Approximation via Information Functionals

Abstract

An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Let $P_{S_n}$ be the distribution of a sum $S_n=\Sumn Y_i$ of independent integer-valued random variables $Y_i$. Nonasymptotic bounds are derived for the distance between $P_{S_n}$ and an appropriately chosen compound Poisson law. In the case where all $Y_i$ have the same conditional distribution given $\{Y_i\ne 0\}$, a bound on the relative entropy distance between $P_{S_n}$ and the compound Poisson distribution is derived, based on the data-processing property of relative entropy and earlier Poisson approximation results. When the $Y_i$ have arbitrary distributions, corresponding bounds are derived in terms of the total variation distance. The main technical ingredient is the introduction of two "information functionals," and the analysis of their properties. These information functionals play a role analogous to that of the classical Fisher information in normal approximation. Detailed comparisons are made between the resulting inequalities and related bounds.

27 pages

References in corpus (1)

• Log-concavity, ultra-log-concavity, and a maximum entropy property of discrete compound Poisson measures

Cited by in corpus (19)

• Forward and Reverse Entropy Power Inequalities in Convex Geometry
• Log-concavity, ultra-log-concavity, and a maximum entropy property of discrete compound Poisson measures
• Entropy Bounds for Discrete Random Variables via Maximal Coupling
• Rogozin's convolution inequality for locally compact groups
• Combinatorial Entropy Power Inequalities: A Preliminary Study of the Stam region
• Entropy Inequalities for Sums in Prime Cyclic Groups
• Majorization and Rényi Entropy Inequalities via Sperner Theory
• An Information-Theoretic Perspective of the Poisson Approximation via the Chen-Stein Method
• Entropies of weighted sums in cyclic groups and an application to polar codes
• Discrete Stein characterizations and discrete information distances
• On the Entropy of Sums of Bernoulli Random Variables via the Chen-Stein Method
• A unified approach to Stein characterizations
• Local Pinsker inequalities via Stein's discrete density approach
• Entropy and the fourth moment phenomenon
• Log-Hessian and Deviation Bounds for Markov Semi-Groups, and Regularization Effect in $L^1$
• On a connection between Stein characterizations and Fisher information
• Relative entropy bounds for sampling with and without replacement
• Entropy and thinning of discrete random variables
• Stein's density approach and information inequalities