Generalizations of Fano's Inequality for Conditional Information Measures via Majorization Theory
arXiv:1801.02876 · doi:10.3390/e22030288
Abstract
Fano's inequality is one of the most elementary, ubiquitous, and important tools in information theory. Using majorization theory, Fano's inequality is generalized to a broad class of information measures, which contains those of Shannon and Rényi. When specialized to these measures, it recovers and generalizes the classical inequalities. Key to the derivation is the construction of an appropriate conditional distribution inducing a desired marginal distribution on a countably infinite alphabet. The construction is based on the infinite-dimensional version of Birkhoff's theorem proven by Révész [Acta Math. Hungar. 1962, 3, 188{\textendash}198], and the constraint of maintaining a desired marginal distribution is similar to coupling in probability theory. Using our Fano-type inequalities for Shannon's and Rényi's information measures, we also investigate the asymptotic behavior of the sequence of Shannon's and Rényi's equivocations when the error probabilities vanish. This asymptotic behavior provides a novel characterization of the asymptotic equipartition property (AEP) via Fano's inequality.
arXiv comment44 pages, 3 figures
References in corpus (14)
- Rényi Divergence and Kullback-Leibler Divergence
- General state space Markov chains and MCMC algorithms
- Exponential decreasing rate of leaked information in universal random privacy amplification
- State Amplification
- Entropy Bounds for Discrete Random Variables via Maximal Coupling
- On properties of the space of quantum states and their application to construction of entanglement monotones
- Majorization and Rényi Entropy Inequalities via Sperner Theory
- A Proof of the Strong Converse Theorem for Gaussian Multiple Access Channels
- On Data-Processing and Majorization Inequalities for -Divergences with Applications
- List decoding - random coding exponents and expurgated exponents
- Source and Channel Polarization over Finite Fields and Reed-Solomon Matrices
- Some Properties of Rényi Entropy over Countably Infinite Alphabets
- Sharp Bounds on Arimoto's Conditional Rényi Entropies Between Two Distinct Orders
- On the error probability of stochastic decision and stochastic decoding