Entropies of weighted sums in cyclic groups and an application to polar codes

Abstract

In this note, the following basic question is explored: in a cyclic group, how are the Shannon entropies of the sum and difference of i.i.d. random variables related to each other? For the integer group, we show that they can differ by any real number additively, but not too much multiplicatively; on the other hand, for $\mathbb{Z}/3\mathbb{Z}$, the entropy of the difference is always at least as large as that of the sum. These results are closely related to the study of more-sum-than-difference (i.e. MSTD) sets in additive combinatorics. We also investigate polar codes for $q$-ary input channels using non-canonical kernels to construct the generator matrix, and present applications of our results to constructing polar codes with significantly improved error probability compared to the canonical construction.

References in corpus (2)

• Forward and Reverse Entropy Power Inequalities in Convex Geometry
• Dimensional behaviour of entropy and information

Cited by in corpus (4)

• Majorization and Rényi Entropy Inequalities via Sperner Theory
• A combinatorial approach to small ball inequalities for sums and differences
• Modular Arithmetic Erasure Channels and Their Multilevel Channel Polarization
• Countably Infinite Multilevel Source Polarization for Non-Stationary Erasure Distributions