Inferring Sparsity: Compressed Sensing using Generalized Restricted Boltzmann Machines

Abstract

In this work, we consider compressed sensing reconstruction from $M$ measurements of $K$-sparse structured signals which do not possess a writable correlation model. Assuming that a generative statistical model, such as a Boltzmann machine, can be trained in an unsupervised manner on example signals, we demonstrate how this signal model can be used within a Bayesian framework of signal reconstruction. By deriving a message-passing inference for general distribution restricted Boltzmann machines, we are able to integrate these inferred signal models into approximate message passing for compressed sensing reconstruction. Finally, we show for the MNIST dataset that this approach can be very effective, even for $M<K$.

IEEE Information Theory Workshop, 2016

References in corpus (6)

• A survey of sparse representation: algorithms and applications
• A Deep Learning Approach to Structured Signal Recovery
• Probabilistic Reconstruction in Compressed Sensing: Algorithms, Phase Diagrams, and Threshold Achieving Matrices
• ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Random Measurements
• Advanced Mean Field Theory of Restricted Boltzmann Machine
• Approximate Message Passing with Restricted Boltzmann Machine Priors

Cited by in corpus (5)

• Mean-field message-passing equations in the Hopfield model and its generalizations
• Constrained Low-rank Matrix Estimation: Phase Transitions, Approximate Message Passing and Applications
• Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors
• Restricted Boltzmann Machine, recent advances and mean-field theory
• Inference in Deep Networks in High Dimensions