AMP-Inspired Deep Networks for Sparse Linear Inverse Problems

Abstract

Deep learning has gained great popularity due to its widespread success on many inference problems. We consider the application of deep learning to the sparse linear inverse problem, where one seeks to recover a sparse signal from a few noisy linear measurements. In this paper, we propose two novel neural-network architectures that decouple prediction errors across layers in the same way that the approximate message passing (AMP) algorithms decouple them across iterations: through Onsager correction. First, we propose a "learned AMP" network that significantly improves upon Gregor and LeCun's "learned ISTA." Second, inspired by the recently proposed "vector AMP" (VAMP) algorithm, we propose a "learned VAMP" network that offers increased robustness to deviations in the measurement matrix from i.i.d. Gaussian. In both cases, we jointly learn the linear transforms and scalar nonlinearities of the network. Interestingly, with i.i.d. signals, the linear transforms and scalar nonlinearities prescribed by the VAMP algorithm coincide with the values learned through back-propagation, leading to an intuitive interpretation of learned VAMP. Finally, we apply our methods to two problems from 5G wireless communications: compressive random access and massive-MIMO channel estimation.

arXiv commentto appear in IEEE Transactions in Signal Processing

References in corpus (22)

• Adam: A Method for Stochastic Optimization
• Scaling up MIMO: Opportunities and Challenges with Very Large Arrays
• Message Passing Algorithms for Compressed Sensing
• Training Very Deep Networks
• The dynamics of message passing on dense graphs, with applications to compressed sensing
• Residual Networks Behave Like Ensembles of Relatively Shallow Networks
• Expectation-Maximization Gaussian-Mixture Approximate Message Passing
• A Deep Learning Approach to Structured Signal Recovery
• Compressive Phase Retrieval via Generalized Approximate Message Passing
• Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures
• Universality in polytope phase transitions and message passing algorithms
• Adaptive Damping and Mean Removal for the Generalized Approximate Message Passing Algorithm
• Sparse Signal Processing Concepts for Efficient 5G System Design
• Message-Passing Estimation from Quantized Samples
• Maximal Sparsity with Deep Networks?
• Learning optimal nonlinearities for iterative thresholding algorithms
• Near optimal compressed sensing without priors: Parametric SURE Approximate Message Passing
• Hybrid Approximate Message Passing
• ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Random Measurements
• Multi-Layer Generalized Linear Estimation
• Learning Efficient Structured Sparse Models
• Learning Deep ${\ell }_0$ Encoders

Cited by in corpus (79)

• Model-Driven Deep Learning for MIMO Detection
• AMP-Net: Denoising based Deep Unfolding for Compressive Image Sensing
• Trainable ISTA for Sparse Signal Recovery
• Deep Learning for Beamspace Channel Estimation in Millimeter-Wave Massive MIMO Systems
• COAST: COntrollable Arbitrary-Sampling NeTwork for Compressive Sensing
• Content-aware Scalable Deep Compressed Sensing
• Deep BCD-Net Using Identical Encoding-Decoding CNN Structures for Iterative Image Recovery
• Learned Conjugate Gradient Descent Network for Massive MIMO Detection
• Learning to Optimize: A Primer and A Benchmark
• Deep Learning Methods for Solving Linear Inverse Problems: Research Directions and Paradigms
• Advance Warning Methodologies for COVID-19 using Chest X-Ray Images
• Learning to Continuously Optimize Wireless Resource in a Dynamic Environment: A Bilevel Optimization Perspective
• Deep Learning for Channel Sensing and Hybrid Precoding in TDD Massive MIMO OFDM Systems
• Plug-in Estimation in High-Dimensional Linear Inverse Problems: A Rigorous Analysis
• VPNet: Variable Projection Networks
• Solving Sparse Linear Inverse Problems in Communication Systems: A Deep Learning Approach With Adaptive Depth
• ADMM-Net for Communication Interference Removal in Stepped-Frequency Radar
• Deep Learning for Physical-Layer 5G Wireless Techniques: Opportunities, Challenges and Solutions
• MIMO-OFDM-Based Massive Connectivity With Frequency Selectivity Compensation
• Hybrid ISTA: Unfolding ISTA With Convergence Guarantees Using Free-Form Deep Neural Networks
• Structured LISTA for Multidimensional Harmonic Retrieval
• Implicit differentiation of Lasso-type models for hyperparameter optimization
• Plug-and-Play Quantum Adaptive Denoiser for Deconvolving Poisson Noisy Images
• DDPG-Driven Deep-Unfolding with Adaptive Depth for Channel Estimation with Sparse Bayesian Learning
• ATASI-Net: An Efficient Sparse Reconstruction Network for Tomographic SAR Imaging with Adaptive Threshold
• Learning to Continuously Optimize Wireless Resource In Episodically Dynamic Environment
• Deep Coupled-Representation Learning for Sparse Linear Inverse Problems with Side Information
• Antenna Array Diagnosis for Millimeter-Wave MIMO Systems
• Representation Based Regression for Object Distance Estimation
• Nest-DGIL: Nesterov-optimized Deep Geometric Incremental Learning for CS Image Reconstruction
• Plug-and-Play Regularization using Linear Solvers
• A Survey on Deep-Learning based Techniques for Modeling and Estimation of MassiveMIMO Channels
• Deep unfolding of the weighted MMSE beamforming algorithm
• Deep Learning Techniques for Inverse Problems in Imaging
• Theoretical Interpretation of Learned Step Size in Deep-Unfolded Gradient Descent
• Layered Belief Propagation for Low-complexity Large MIMO Detection Based on Statistical Beams
• ISTA-Net++: Flexible Deep Unfolding Network for Compressive Sensing
• DECONET: an Unfolding Network for Analysis-based Compressed Sensing with Generalization Error Bounds
• Deep Learning for Wireless Communications: An Emerging Interdisciplinary Paradigm
• Iterative Algorithm Induced Deep-Unfolding Neural Networks: Precoding Design for Multiuser MIMO Systems
• ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing
• Model-Driven Deep Learning Based Channel Estimation and Feedback for Millimeter-Wave Massive Hybrid MIMO Systems
• Learnable Descent Algorithm for Nonsmooth Nonconvex Image Reconstruction
• DuRIN: A Deep-unfolded Sparse Seismic Reflectivity Inversion Network
• Convolutional Sparse Support Estimator Network (CSEN) From energy efficient support estimation to learning-aided Compressive Sensing
• A Model-Driven Deep Learning Network for MIMO Detection
• The Global Optimization Geometry of Shallow Linear Neural Networks
• Learning step sizes for unfolded sparse coding
• Dynamic Upsampling of Smoke through Dictionary-based Learning
• Multimodal Deep Unfolding for Guided Image Super-Resolution
• Deep Learning Techniques for Compressive Sensing-Based Reconstruction and Inference -- A Ubiquitous Systems Perspective
• Deep Griffin-Lim Iteration
• A Flexible Convolutional Solver with Application to Photorealistic Style Transfer
• Approximate Message Passing for Indoor THz Channel Estimation
• ADMM-based Decoder for Binary Linear Codes Aided by Deep Learning
• A Model-Driven Deep Learning Method for Massive MIMO Detection
• Algorithm Unrolling for Massive Access via Deep Neural Network with Theoretical Guarantee
• Phase Retrieval using Expectation Consistent Signal Recovery Algorithm based on Hypernetwork
• Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for mmWave Multiuser MIMO with Lens Arrays
• A Novel Learnable Gradient Descent Type Algorithm for Non-convex Non-smooth Inverse Problems
• Scalable Deep Compressive Sensing
• Solving inverse problems via auto-encoders
• Deep-Learned Approximate Message Passing for Asynchronous Massive Connectivity
• Feature-Aided Adaptive-Tuning Deep Learning for Massive Device Detection
• Performance Analysis of Plug-and-Play ADMM: A Graph Signal Processing Perspective
• Learning Generative Prior with Latent Space Sparsity Constraints
• Operational Support Estimator Networks
• MC-ISTA-Net: Adaptive Measurement and Initialization and Channel Attention Optimization inspired Neural Network for Compressive Sensing
• Explicit CSI Feedback Compression via Learned Approximate Message Passing
• Bayes-Optimal Convolutional AMP
• Learned Interpretable Residual Extragradient ISTA for Sparse Coding
• Block-Sparse Recovery Network for Two-Dimensional Harmonic Retrieval
• Learning to solve TV regularized problems with unrolled algorithms
• Compressed Sensing via Measurement-Conditional Generative Models
• DeepFPC: Deep Unfolding of a Fixed-Point Continuation Algorithm for Sparse Signal Recovery from Quantized Measurements
• Fast Super-resolution 3D SAR Imaging Using an Unfolded Deep Network
• Tree Search Network for Sparse Regression
• Deep Learning for Cross-Technology Communication Design
• Theoretical Linear Convergence of Deep Unfolding Network for Block-Sparse Signal Recovery