Entropy bounds on abelian groups and the Ruzsa divergence

Abstract

Over the past few years, a family of interesting new inequalities for the entropies of sums and differences of random variables has been developed by Ruzsa, Tao and others, motivated by analogous results in additive combinatorics. The present work extends these earlier results to the case of random variables taking values in ${\mathbb{R}}^n$ or, more generally, in arbitrary locally compact and Polish abelian groups. We isolate and study a key quantity, the Ruzsa divergence between two probability distributions, and we show that its properties can be used to extend the earlier inequalities to the present general setting. The new results established include several variations on the theme that the entropies of the sum and the difference of two independent random variables severely constrain each other. Although the setting is quite general, the result are already of interest (and new) for random vectors in ${\mathbb{R}}^n$. In that special case, quantitative bounds are provided for the stability of the equality conditions in the entropy power inequality; a reverse entropy power inequality for log-concave random vectors is proved; an information-theoretic analog of the Rogers-Shephard inequality for convex bodies is established; and it is observed that some of these results lead to new inequalities for the determinants of positive-definite matrices. Moreover, by considering the multiplicative subgroups of the complex plane, one obtains new inequalities for the differential entropies of products and ratios of nonzero, complex-valued random variables.

26 pages. Changes in v2: Added Section V on entropies of products of random variables, corrected several typos, added some references

References in corpus (9)

• Forward and Reverse Entropy Power Inequalities in Convex Geometry
• Dimensional behaviour of entropy and information
• On an Inequality of Karlin and Rinott Concerning Weighted Sums of i.i.d. Random Variables
• Rogozin's convolution inequality for locally compact groups
• Combinatorial Entropy Power Inequalities: A Preliminary Study of the Stam region
• Entropy Inequalities for Sums in Prime Cyclic Groups
• The convexification effect of Minkowski summation
• New Upper Bound for Sums of Dilates
• Propagation of Localization Optimal Entropy Production and Convergence rates for the Central Limit Theorem

Cited by in corpus (23)

• Forward and Reverse Entropy Power Inequalities in Convex Geometry
• A lower bound on the differential entropy of log-concave random vectors with applications
• Rogozin's convolution inequality for locally compact groups
• Combinatorial Entropy Power Inequalities: A Preliminary Study of the Stam region
• Entropy Inequalities for Sums in Prime Cyclic Groups
• Majorization and Rényi Entropy Inequalities via Sperner Theory
• The Differential Entropy of Mixtures: New Bounds and Applications
• Sharp moment-entropy inequalities and capacity bounds for log-concave distributions
• Two remarks on generalized entropy power inequalities
• Concentration of information content for convex measures
• On the volume of the Minkowski sum of zonoids
• Stability of the Shannon-Stam inequality via the Föllmer process
• Volumes of subset Minkowski sums and the Lyusternik region
• Conditional Rényi entropy and the relationships between Rényi capacities
• Quantum Ruzsa Divergence to Quantify Magic
• A combinatorial approach to small ball inequalities for sums and differences
• The convexification effect of Minkowski summation
• Reversals of Rényi Entropy Inequalities under Log-Concavity
• Entropic additive energy and entropy inequalities for sums and products
• Sumset estimates in convex geometry
• Weighted Brunn-Minkowski Theory II: Inequalities for Mixed Measures and Applications
• Equivalence of additive-combinatorial linear inequalities for Shannon entropy and differential entropy
• Countably Infinite Multilevel Source Polarization for Non-Stationary Erasure Distributions