Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization
arXiv:1501.01797
Abstract
Generalized Linear Models (GLMs), where a random vector is observed through a noisy, possibly nonlinear, function of a linear transform arise in a range of applications in nonlinear filtering and regression. Approximate Message Passing (AMP) methods, based on loopy belief propagation, are a promising class of approaches for approximate inference in these models. AMP methods are computationally simple, general, and admit precise analyses with testable conditions for optimality for large i.i.d. transforms . However, the algorithms can easily diverge for general . This paper presents a convergent approach to the generalized AMP (GAMP) algorithm based on direct minimization of a large-system limit approximation of the Bethe Free Energy (LSL-BFE). The proposed method uses a double-loop procedure, where the outer loop successively linearizes the LSL-BFE and the inner loop minimizes the linearized LSL-BFE using the Alternating Direction Method of Multipliers (ADMM). The proposed method, called ADMM-GAMP, is similar in structure to the original GAMP method, but with an additional least-squares minimization. It is shown that for strictly convex, smooth penalties, ADMM-GAMP is guaranteed to converge to a local minima of the LSL-BFE, thus providing a convergent alternative to GAMP that is stable under arbitrary transforms. Simulations are also presented that demonstrate the robustness of the method for non-convex penalties as well.
References in corpus (11)
- Message Passing Algorithms for Compressed Sensing
- Expectation-Maximization Gaussian-Mixture Approximate Message Passing
- Bilinear Generalized Approximate Message Passing
- Compressive Imaging using Approximate Message Passing and a Markov-Tree Prior
- Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing
- Adaptive Damping and Mean Removal for the Generalized Approximate Message Passing Algorithm
- Message-Passing Estimation from Quantized Samples
- Belief Optimization for Binary Networks: A Stable Alternative to Loopy Belief Propagation
- Generalized Approximate Message Passing for Estimation with Random Linear Mixing
- An Empirical-Bayes Approach to Recovering Linearly Constrained Non-Negative Sparse Signals
- Sparse Estimation with the Swept Approximated Message-Passing Algorithm
Cited by in corpus (5)
- Statistical physics of inference: Thresholds and algorithms
- Compressive Imaging via Approximate Message Passing with Image Denoising
- An Overview of Multi-Processor Approximate Message Passing
- Approximate cross-validation formula for Bayesian linear regression
- Approximate Message Passing in Coded Aperture Snapshot Spectral Imaging