Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation
arXiv:2005.03991
Abstract
Un-trained convolutional neural networks have emerged as highly successful tools for image recovery and restoration. They are capable of solving standard inverse problems such as denoising and compressive sensing with excellent results by simply fitting a neural network model to measurements from a single image or signal without the need for any additional training data. For some applications, this critically requires additional regularization in the form of early stopping the optimization. For signal recovery from a few measurements, however, un-trained convolutional networks have an intriguing self-regularizing property: Even though the network can perfectly fit any image, the network recovers a natural image from few measurements when trained with gradient descent until convergence. In this paper, we provide numerical evidence for this property and study it theoretically. We show that---without any further regularization---an un-trained convolutional neural network can approximately reconstruct signals and images that are sufficiently structured, from a near minimal number of random measurements.
arXiv commentarXiv admin note: text overlap with arXiv:1910.14634
References in corpus (14)
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
- fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
- Compressed Sensing using Generative Models
- Compressed Sensing with Deep Image Prior and Learned Regularization
- Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks
- Towards moderate overparameterization: global convergence guarantees for training shallow neural networks
- Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators
- Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?
- Spurious Valleys in Two-layer Neural Network Optimization Landscapes
- Generative Models for Low-Rank Video Representation and Reconstruction
- Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian
- Robust Compressive Phase Retrieval via Deep Generative Priors
- A Provably Convergent Scheme for Compressive Sensing under Random Generative Priors
- Regularizing linear inverse problems with convolutional neural networks