Entropies of weighted sums in cyclic groups and an application to polar codes
arXiv:1512.00135 · doi:10.3390/e19090235
Abstract
In this note, the following basic question is explored: in a cyclic group, how are the Shannon entropies of the sum and difference of i.i.d. random variables related to each other? For the integer group, we show that they can differ by any real number additively, but not too much multiplicatively; on the other hand, for , the entropy of the difference is always at least as large as that of the sum. These results are closely related to the study of more-sum-than-difference (i.e. MSTD) sets in additive combinatorics. We also investigate polar codes for -ary input channels using non-canonical kernels to construct the generator matrix, and present applications of our results to constructing polar codes with significantly improved error probability compared to the canonical construction.
References in corpus (17)
- Channel polarization: A method for constructing capacity-achieving codes for symmetric binary-input memoryless channels
- Source Polarization
- Information Inequalities for Joint Distributions, with Interpretations and Applications
- Finite-Length Scaling of Polar Codes
- Sumset and inverse sumset theorems for Shannon entropy
- Forward and Reverse Entropy Power Inequalities in Convex Geometry
- Sumset and Inverse Sumset Inequalities for Differential Entropy and Mutual Information
- Log-concavity, ultra-log-concavity, and a maximum entropy property of discrete compound Poisson measures
- Compound Poisson Approximation via Information Functionals
- Thinning, Entropy and the Law of Thin Numbers
- Monotonicity, thinning and discrete versions of the Entropy Power Inequality
- Constructing MSTD Sets Using Bidirectional Ballot Sequences
- Entropy and set cardinality inequalities for partition-determined functions
- Dimensional behaviour of entropy and information
- Some explicit constructions of sets with more sums than differences
- Degrees of freedom in vector interference channels
- Counting MSTD Sets in Finite Abelian Groups
Cited by in corpus (4)
- Majorization and Rényi Entropy Inequalities via Sperner Theory
- A combinatorial approach to small ball inequalities for sums and differences
- Modular Arithmetic Erasure Channels and Their Multilevel Channel Polarization
- Countably Infinite Multilevel Source Polarization for Non-Stationary Erasure Distributions