Multi-Layer Generalized Linear Estimation
arXiv:1701.06981 · doi:10.1109/ISIT.2017.8006899
Abstract
We consider the problem of reconstructing a signal from multi-layered (possibly) non-linear measurements. Using non-rigorous but standard methods from statistical physics we present the Multi-Layer Approximate Message Passing (ML-AMP) algorithm for computing marginal probabilities of the corresponding estimation problem and derive the associated state evolution equations to analyze its performance. We also give the expression of the asymptotic free energy and the minimal information-theoretically achievable reconstruction error. Finally, we present some applications of this measurement model for compressed sensing and perceptron learning with structured matrices/patterns, and for a simple model of estimation of latent variables in an auto-encoder.
arXiv comment5 pages, 1 figure
References in corpus (6)
- Message Passing Algorithms for Compressed Sensing
- Community Structure in Time-Dependent, Multiscale, and Multiplex Networks
- The dynamics of message passing on dense graphs, with applications to compressed sensing
- Probabilistic Reconstruction in Compressed Sensing: Algorithms, Phase Diagrams, and Threshold Achieving Matrices
- Analysis of CDMA systems that are characterized by eigenvalue spectrum
- Inference from correlated patterns: a unified theory for perceptron learning and linear vector channels
Cited by in corpus (6)
- AMP-Inspired Deep Networks for Sparse Linear Inverse Problems
- Entropy and mutual information in models of deep neural networks
- The Mutual Information in Random Linear Estimation Beyond i.i.d. Matrices
- Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (or : How to Prove Kabashima's Replica Formula)
- Multi-Layer Bilinear Generalized Approximate Message Passing
- Deep learning via message passing algorithms based on belief propagation