Exploiting Statistical Dependencies in Sparse Representations for Signal Recovery

Abstract

Signal modeling lies at the core of numerous signal and image processing applications. A recent approach that has drawn considerable attention is sparse representation modeling, in which the signal is assumed to be generated as a combination of a few atoms from a given dictionary. In this work we consider a Bayesian setting and go beyond the classic assumption of independence between the atoms. The main goal of this paper is to introduce a statistical model that takes such dependencies into account and show how this model can be used for sparse signal recovery. We follow the suggestion of two recent works and assume that the sparsity pattern is modeled by a Boltzmann machine, a commonly used graphical model. For general dependency models, exact MAP and MMSE estimation of the sparse representation becomes computationally complex. To simplify the computations, we propose greedy approximations of the MAP and MMSE estimators. We then consider a special case in which exact MAP is feasible, by assuming that the dictionary is unitary and the dependency model corresponds to a certain sparse graph. Exploiting this structure, we develop an efficient message passing algorithm that recovers the underlying signal. When the model parameters defining the underlying graph are unknown, we suggest an algorithm that learns these parameters directly from the data, leading to an iterative scheme for adaptive sparse signal recovery. The effectiveness of our approach is demonstrated on real-life signals - patches of natural images - where we compare the denoising performance to that of previous recovery methods that do not exploit the statistical dependencies.

18 pages, 6 figures, to appear in IEEE Trans. on Signal Processing

References in corpus (5)

• Compressed Sensing of Block-Sparse Signals: Uncertainty Relations and Efficient Recovery
• Structured Compressed Sensing: From Theory to Applications
• Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors
• On MMSE and MAP Denoising Under Sparse Representation Modeling Over a Unitary Dictionary
• Partially Linear Estimation with Application to Sparse Signal Recovery From Measurement Pairs

Cited by in corpus (17)

• Structured Compressed Sensing: From Theory to Applications
• Extension of SBL Algorithms for the Recovery of Block Sparse Signals with Intra-Block Correlation
• Compressed Sensing for Energy-Efficient Wireless Telemonitoring of Noninvasive Fetal ECG via Block Sparse Bayesian Learning
• Two-Dimensional Pattern-Coupled Sparse Bayesian Learning via Generalized Approximate Message Passing
• Projection Design For Statistical Compressive Sensing: A Tight Frame Based Approach
• Exploiting Restricted Boltzmann Machines and Deep Belief Networks in Compressed Sensing
• Variance State Propagation for Structured Sparse Bayesian Learning
• Application of Compressive Sensing Techniques in Distributed Sensor Networks: A Survey
• Greedy Sparsity-Promoting Algorithms for Distributed Learning
• Learning efficient sparse and low rank models
• An Adaptive Markov Random Field for Structured Compressive Sensing
• Structural Sparsity in Multiple Measurements
• Power-Constrained Sparse Gaussian Linear Dimensionality Reduction over Noisy Channels
• Structured Sparsity Models for Multiparty Speech Recovery from Reverberant Recordings
• Multi-Focus Image Fusion Using Sparse Representation and Coupled Dictionary Learning
• Peculiar spectral statistics of ensembles of trees and star-like graphs
• Sparse Signal Recovery in the Presence of Intra-Vector and Inter-Vector Correlation