The Mutual Information In The Vicinity of Capacity-Achieving Input Distributions

Abstract

The mutual information is bounded from above by a decreasing affine function of the square of the distance between the input distribution and the set of all capacity-achieving input distributions ${\Pi }_{\mathcal{A}}$, on small enough neighborhoods of ${\Pi }_{\mathcal{A}}$, using an identity due to Topsøe and the Pinsker's inequality, assuming that the input set of the channel is finite and the constraint set $\mathcal{A}$ is polyhedral, i.e., can be described by (possibly multiple but) finitely many linear constraints. Counterexamples demonstrating nonexistence of such a quadratic bound are provided for the case of infinitely many linear constraints and the case of infinite input sets. Using Taylor's theorem with the remainder term, rather than the Pinsker's inequality and invoking Moreau's decomposition theorem the exact characterization of the slowest decrease of the mutual information with the distance to ${\Pi }_{\mathcal{A}}$ is determined on small neighborhoods of ${\Pi }_{\mathcal{A}}$. Corresponding results for classical-quantum channels are established under separable output Hilbert space assumption for the quadratic bound and under finite-dimensional output Hilbert space assumption for the exact characterization. Implications of these observations for the channel coding problem and applications of the proof techniques to related problems are discussed.

18 pages, presented at ISIT 2023, submitted to IEEE Transactions on Information Theory on June 13, 2023, revised on September 6, 2024, accepted on April 05, 2025

References in corpus (12)

• Information Spectrum Approach to Second-Order Coding Rate in Channel Coding
• Quantum Information Processing with Finite Resources -- Mathematical Foundations
• Fixed-length lossy compression in the finite blocklength regime
• A Tight Upper Bound for the Third-Order Asymptotics for Most Discrete Memoryless Channels
• Second-Order Asymptotics for the Classical Capacity of Image-Additive Quantum Channels
• Complete entropic inequalities for quantum Markov chains
• Quantum Sphere-Packing Bounds with Polynomial Prefactors
• The Renyi Capacity and Center
• Second-Order Asymptotics for the Gaussian MAC with Degraded Message Sets
• Quantum $f$-divergences in von Neumann algebras I. Standard $f$-divergences
• The Augustin Capacity and Center
• Comments on "Channel Coding Rate in the Finite Blocklength Regime": On the Quadratic Decaying Property of the Information Rate Function