Optimal Quantization for Distribution Synthesis

Abstract

Finite precision approximations of discrete probability distributions are considered, applicable for distribution synthesis, e.g., probabilistic shaping. Two algorithms are presented that find the optimal $M$-type approximation $Q$ of a distribution $P$ in terms of the variational distance $|Q-P{|}_1$ and the informational divergence $\mathbb{D}(Q|P)$. Bounds on the approximation errors are derived and shown to be asymptotically tight. Several examples illustrate that the variational distance optimal approximation can be quite different from the informational divergence optimal approximation.

Submitted to the IEEE Transactions on Information Theory

References in corpus (4)

• Rényi Divergence and Kullback-Leibler Divergence
• $f$-divergence Inequalities
• Optimal Non-Uniform Mapping for Probabilistic Shaping
• Greedy Algorithms for Optimal Distribution Approximation

Cited by in corpus (8)

• $f$-divergence Inequalities
• Probabilistic Shaping for Finite Blocklengths: Distribution Matching and Sphere Shaping
• End-to-end Optimization of Constellation Shaping for Wiener Phase Noise Channels with a Differentiable Blind Phase Search
• Multilevel Binary Polar-Coded Modulation Achieving the Capacity of Asymmetric Channels
• A Note on Reverse Pinsker Inequalities
• Divergence Scaling of Fixed-Length, Binary-Output, One-to-One Distribution Matching
• Tight lower bound on the error exponent of classical-quantum channels
• Greedy Algorithms for Optimal Distribution Approximation