Approximate Message Passing with Restricted Boltzmann Machine Priors

Abstract

Approximate Message Passing (AMP) has been shown to be an excellent statistical approach to signal inference and compressed sensing problem. The AMP framework provides modularity in the choice of signal prior; here we propose a hierarchical form of the Gauss-Bernouilli prior which utilizes a Restricted Boltzmann Machine (RBM) trained on the signal support to push reconstruction performance beyond that of simple iid priors for signals whose support can be well represented by a trained binary RBM. We present and analyze two methods of RBM factorization and demonstrate how these affect signal reconstruction performance within our proposed algorithm. Finally, using the MNIST handwritten digit dataset, we show experimentally that using an RBM allows AMP to approach oracle-support performance.

References in corpus (2)

• Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
• Probabilistic Reconstruction in Compressed Sensing: Algorithms, Phase Diagrams, and Threshold Achieving Matrices

Cited by in corpus (13)

• Statistical physics of inference: Thresholds and algorithms
• Machine Learning Spatial Geometry from Entanglement Features
• Mean-field inference methods for neural networks
• Exploiting Restricted Boltzmann Machines and Deep Belief Networks in Compressed Sensing
• Intensity-only optical compressive imaging using a multiply scattering material and a double phase retrieval approach
• Inferring Sparsity: Compressed Sensing using Generalized Restricted Boltzmann Machines
• A Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines
• Robust phase retrieval with the swept approximate message passing (prSAMP) algorithm
• Free energies of Boltzmann Machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit
• A relativistic extension of Hopfield neural networks via the mechanical analogy
• A Variational Bayesian Approach for Image Restoration. Application to Image Deblurring with Poisson-Gaussian Noise
• Compressed Sensing via Measurement-Conditional Generative Models
• Tree Search Network for Sparse Regression