The Mutual Information in Random Linear Estimation Beyond i.i.d. Matrices
arXiv:1802.08963 · doi:10.1109/ISIT.2018.8437522
Abstract
There has been definite progress recently in proving the variational single-letter formula given by the heuristic replica method for various estimation problems. In particular, the replica formula for the mutual information in the case of noisy linear estimation with random i.i.d. matrices, a problem with applications ranging from compressed sensing to statistics, has been proven rigorously. In this contribution we go beyond the restrictive i.i.d. matrix assumption and discuss the formula proposed by Takeda, Uda, Kabashima and later by Tulino, Verdu, Caire and Shamai who used the replica method. Using the recently introduced adaptive interpolation method and random matrix theory, we prove this formula for a relevant large sub-class of rotationally invariant matrices.
arXiv commentPresented at the 2018 IEEE International Symposium on Information Theory (ISIT)
References in corpus (10)
- Message Passing Algorithms for Compressed Sensing
- Statistical physics of inference: Thresholds and algorithms
- The Thermodynamic Limit in Mean Field Spin Glass Models
- Statistical physics-based reconstruction in compressed sensing
- Approximate message-passing decoder and capacity-achieving sparse superposition codes
- Analysis of CDMA systems that are characterized by eigenvalue spectrum
- The Mutual Information in Random Linear Estimation
- A note on classical ground state energies
- Inference from correlated patterns: a unified theory for perceptron learning and linear vector channels
- Multi-Layer Generalized Linear Estimation
Cited by in corpus (5)
- The adaptive interpolation method for proving replica formulas. Applications to the Curie-Weiss and Wigner spike models
- On the Universality of Noiseless Linear Estimation with Respect to the Measurement Matrix
- Macroscopic Analysis of Vector Approximate Message Passing in a Model Mismatch Setting
- Strong replica symmetry for high-dimensional disordered log-concave Gibbs measures
- Concentration of multi-overlaps for random ferromagnetic spin models