Replica Analysis for Generalized Linear Regression with IID Row Prior
arXiv:2109.12357
Abstract
Different from a typical independent identically distributed (IID) element assumption, this paper studies the estimation of IID row random matrix for the generalized linear model constructed by a linear mixing space and a row-wise mapping channel. The objective inference problem arises in many engineering fields, such as wireless communications, compressed sensing, and phase retrieval. We apply the replica method from statistical mechanics to analyze the exact minimum mean square error (MMSE) under the Bayes-optimal setting, in which the explicit replica symmetric solution of the exact MMSE estimator is obtained. Meanwhile, the input-output mutual information relation between the objective model and the equivalent single-vector system is established. To estimate the signal, we also propose a computationally efficient message passing based algorithm on expectation propagation (EP) perspective and analyze its dynamics. We verify that the asymptotic MSE of proposed algorithm predicted by its state evolution (SE) matches perfectly the exact MMSE estimator predicted by the replica method. That indicates, the optimal MSE error can be attained by the proposed algorithm if it has a unique fixed point.
arXiv comment17 pages, 8 figures
References in corpus (14)
- Message Passing Algorithms for Compressed Sensing
- The large deviation approach to statistical mechanics
- Massive Connectivity with Massive MIMO-Part I: Device Activity Detection and Channel Estimation
- Statistical physics of inference: Thresholds and algorithms
- Bayes-Optimal Joint Channel-and-Data Estimation for Massive MIMO with Low-Precision ADCs
- Compressive Phase Retrieval via Generalized Approximate Message Passing
- Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing
- Phase transitions and sample complexity in Bayes-optimal matrix factorization
- Hybrid Approximate Message Passing
- Analysis of Regularized LS Reconstruction and Random Matrix Ensembles in Compressed Sensing
- A basic introduction to large deviations: Theory, applications, simulations
- Multi-Layer Bilinear Generalized Approximate Message Passing
- Concise Derivation of Complex Bayesian Approximate Message Passing via Expectation Propagation
- Statistical physics of linear and bilinear inference problems