Wyner's Common Information under Rényi Divergence Measures
arXiv:1709.02168
Abstract
We study a generalized version of Wyner's common information problem (also coined the distributed source simulation problem). The original common information problem consists in understanding the minimum rate of the common input to independent processors to generate an approximation of a joint distribution when the distance measure used to quantify the discrepancy between the synthesized and target distributions is the normalized relative entropy. Our generalization involves changing the distance measure to the unnormalized and normalized Rényi divergences of order . We show that the minimum rate needed to ensure the Rényi divergences between the distribution induced by a code and the target distribution vanishes remains the same as the one in Wyner's setting, except when the order . This implies that Wyner's common information is rather robust to the choice of distance measure employed. As a byproduct of the proofs used to the establish the above results, the exponential strong converse for the common information problem under the total variation distance measure is established.
arXiv comment17 pages, 1 figure. The achievability part for in Theorem 1 is incorrect. It was corrected in arXiv:1810.02534
References in corpus (8)
- Rényi Divergence and Kullback-Leibler Divergence
- Coordination Capacity
- Distributed Channel Synthesis
- A Large-Deviation Analysis of the Maximum-Likelihood Learning of Markov Tree Structures
- Exact Random Coding Secrecy Exponents for the Wiretap Channel
- Quantum Achievability Proof via Collision Relative Entropy
- Exponent Function for Source Coding with Side Information at the Decoder at Rates below the Rate Distortion Function
- Rényi divergence and the central limit theorem