A Deep Journey into Super-resolution: A survey

Abstract

Deep convolutional networks based super-resolution is a fast-growing field with numerous practical applications. In this exposition, we extensively compare 30+ state-of-the-art super-resolution Convolutional Neural Networks (CNNs) over three classical and three recently introduced challenging datasets to benchmark single image super-resolution. We introduce a taxonomy for deep-learning based super-resolution networks that groups existing methods into nine categories including linear, residual, multi-branch, recursive, progressive, attention-based and adversarial designs. We also provide comparisons between the models in terms of network complexity, memory footprint, model input and output, learning details, the type of network losses and important architectural differences (e.g., depth, skip-connections, filters). The extensive evaluation performed, shows the consistent and rapid growth in the accuracy in the past few years along with a corresponding boost in model complexity and the availability of large-scale datasets. It is also observed that the pioneering methods identified as the benchmark have been significantly outperformed by the current contenders. Despite the progress in recent years, we identify several shortcomings of existing techniques and provide future research directions towards the solution of these open problems.

arXiv commentAccepted in ACM Computing Surveys

References in corpus (13)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• Natural Language Processing (almost) from Scratch
• Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections
• ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
• Towards Better Exploiting Convolutional Neural Networks for Remote Sensing Scene Classification
• The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
• Is the deconvolution layer the same as a convolutional layer?
• Single Image Super-Resolution via Cascaded Multi-Scale Cross Network
• Resolution limits in astronomical images
• MAMNet: Multi-path Adaptive Modulation Network for Image Super-Resolution
• Chaining Identity Mapping Modules for Image Denoising
• Learning Hybrid Sparsity Prior for Image Restoration: Where Deep Learning Meets Sparse Coding

Cited by in corpus (7)

• TomoGAN: Low-Dose Synchrotron X-Ray Tomography with Generative Adversarial Networks
• Spatial-Spectral Residual Network for Hyperspectral Image Super-Resolution
• EventSR: From Asynchronous Events to Image Reconstruction, Restoration, and Super-Resolution via End-to-End Adversarial Learning
• Spatial-Angular Interaction for Light Field Image Super-Resolution
• Differentiable bit-rate estimation for neural-based video codec enhancement
• A Modular Deep Learning Pipeline for Galaxy-Scale Strong Gravitational Lens Detection and Modeling
• Multi-scale super-resolution generation of low-resolution scanned pathological images