Convex Regularization Behind Neural Reconstruction
arXiv:2012.05169
Abstract
Neural networks have shown tremendous potential for reconstructing high-resolution images in inverse problems. The non-convex and opaque nature of neural networks, however, hinders their utility in sensitive applications such as medical imaging. To cope with this challenge, this paper advocates a convex duality framework that makes a two-layer fully-convolutional ReLU denoising network amenable to convex optimization. The convex dual network not only offers the optimum training with convex solvers, but also facilitates interpreting training and prediction. In particular, it implies training neural networks with weight decay regularization induces path sparsity while the prediction is piecewise linear filtering. A range of experiments with MNIST and fastMRI datasets confirm the efficacy of the dual network optimization problem.
References in corpus (16)
- Adam: A Method for Stochastic Optimization
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
- Deep Convolutional Neural Network for Inverse Problems in Imaging
- Neural Tangent Kernel: Convergence and Generalization in Neural Networks
- On instabilities of deep learning in image reconstruction - Does AI come at a cost?
- Medical image denoising using convolutional denoising autoencoders
- fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
- Denoising Prior Driven Deep Neural Network for Image Restoration
- Input Convex Neural Networks
- Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks
- Training Neural Networks with Local Error Signals
- Unrolled Optimization with Deep Priors
- Neural Proximal Gradient Descent for Compressive Imaging
- Learned convex regularizers for inverse problems
- Revealing the Structure of Deep Neural Networks via Convex Duality
- Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks
Cited by in corpus (4)
- Neural Spectrahedra and Semidefinite Lifts: Global Convex Optimization of Polynomial Activation Neural Networks in Fully Polynomial-Time
- Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions
- Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithms
- The Hidden Convex Optimization Landscape of Two-Layer ReLU Neural Networks: an Exact Characterization of the Optimal Solutions