Deep Learning for Inverse Problems: Bounds and Regularizers

Abstract

Inverse problems arise in a number of domains such as medical imaging, remote sensing, and many more, relying on the use of advanced signal and image processing approaches -- such as sparsity-driven techniques -- to determine their solution. This paper instead studies the use of deep learning approaches to approximate the solution of inverse problems. In particular, the paper provides a new generalization bound, depending on key quantity associated with a deep neural network -- its Jacobian matrix -- that also leads to a number of computationally efficient regularization strategies applicable to inverse problems. The paper also tests the proposed regularization strategies in a number of inverse problems including image super-resolution ones. Our numerical results conducted on various datasets show that both fully connected and convolutional neural networks regularized using the regularization or proxy regularization strategies originating from our theory exhibit much better performance than deep networks regularized with standard approaches such as weight-decay.

References in corpus (8)

• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Robust Large Margin Deep Neural Networks
• Spectral Norm Regularization for Improving the Generalizability of Deep Learning
• All you need is a good init
• Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks
• On orthogonality and learning recurrent networks with long term dependencies
• Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks
• Implicit Regularization in Deep Learning

Cited by in corpus (2)

• Normalization Techniques in Training DNNs: Methodology, Analysis and Application
• Controllable Orthogonalization in Training DNNs