Super-Resolution with Deep Adaptive Image Resampling

Abstract

Deep learning based methods have recently pushed the state-of-the-art on the problem of Single Image Super-Resolution (SISR). In this work, we revisit the more traditional interpolation-based methods, that were popular before, now with the help of deep learning. In particular, we propose to use a Convolutional Neural Network (CNN) to estimate spatially variant interpolation kernels and apply the estimated kernels adaptively to each position in the image. The whole model is trained in an end-to-end manner. We explore two ways to improve the results for the case of large upscaling factors, and propose a recursive extension of our basic model. This achieves results that are on par with state-of-the-art methods. We visualize the estimated adaptive interpolation kernels to gain more insight on the effectiveness of the proposed method. We also extend the method to the task of joint image filtering and again achieve state-of-the-art performance.

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Deep Residual Learning for Image Recognition
• Multi-Scale Context Aggregation by Dilated Convolutions
• Going Deeper with Convolutions
• Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections
• Fully Convolutional Networks for Semantic Segmentation
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Deformable Convolutional Networks
• Unsupervised Learning for Physical Interaction through Video Prediction
• Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
• Video Frame Interpolation via Adaptive Separable Convolution

Cited by in corpus (2)

• U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instrument
• NAS-DIP: Learning Deep Image Prior with Neural Architecture Search