End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks

Abstract

One impressive advantage of convolutional neural networks (CNNs) is their ability to automatically learn feature representation from raw pixels, eliminating the need for hand-designed procedures. However, recent methods for single image super-resolution (SR) fail to maintain this advantage. They utilize CNNs in two decoupled steps, i.e., first upsampling the low resolution (LR) image to the high resolution (HR) size with hand-designed techniques (e.g., bicubic interpolation), and then applying CNNs on the upsampled LR image to reconstruct HR results. In this paper, we seek an alternative and propose a new image SR method, which jointly learns the feature extraction, upsampling and HR reconstruction modules, yielding a completely end-to-end trainable deep CNN. As opposed to existing approaches, the proposed method conducts upsampling in the latent feature space with filters that are optimized for the task of image SR. In addition, the HR reconstruction is performed in a multi-scale manner to simultaneously incorporate both short- and long-range contextual information, ensuring more accurate restoration of HR images. To facilitate network training, a new training approach is designed, which jointly trains the proposed deep network with a relatively shallow network, leading to faster convergence and more superior performance. The proposed method is extensively evaluated on widely adopted data sets and improves the performance of state-of-the-art methods with a considerable margin. Moreover, in-depth ablation studies are conducted to verify the contribution of different network designs to image SR, providing additional insights for future research.

References in corpus (3)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Deep Networks for Image Super-Resolution with Sparse Prior

Cited by in corpus (7)

• Convolutional Neural Network-Based Block Up-sampling for Intra Frame Coding
• Super Resolution Convolutional Neural Network Models for Enhancing Resolution of Rock Micro-CT Images
• Super-Resolution via Deep Learning
• Boosting Resolution and Recovering Texture of micro-CT Images with Deep Learning
• Attribute-Guided Face Generation Using Conditional CycleGAN
• GUN: Gradual Upsampling Network for Single Image Super-Resolution
• DeepDPM: Dynamic Population Mapping via Deep Neural Network