Convolutional Neural Networks Analyzed via Inverse Problem Theory and Sparse Representations
arXiv:1807.07998 · doi:10.1049/iet-spr.2018.5220
Abstract
Inverse problems in imaging such as denoising, deblurring, superresolution (SR) have been addressed for many decades. In recent years, convolutional neural networks (CNNs) have been widely used for many inverse problem areas. Although their indisputable success, CNNs are not mathematically validated as to how and what they learn. In this paper, we prove that during training, CNN elements solve for inverse problems which are optimum solutions stored as CNN neuron filters. We discuss the necessity of mutual coherence between CNN layer elements in order for a network to converge to the optimum solution. We prove that required mutual coherence can be provided by the usage of residual learning and skip connections. We have set rules over training sets and depth of networks for better convergence, i.e. performance.
arXiv commentPostPrint IET Signal Processing Journal
References in corpus (9)
- Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
- Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration
- Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections
- Understanding Deep Convolutional Networks
- A Review of Convolutional Neural Networks for Inverse Problems in Imaging
- Working Locally Thinking Globally: Theoretical Guarantees for Convolutional Sparse Coding
- Super-Resolution with Deep Convolutional Sufficient Statistics
- Convolutional Neural Networks Analyzed via Convolutional Sparse Coding
- Learning Efficient Structured Sparse Models