Deep Learning for Single Image Super-Resolution: A Brief Review

Abstract

Single image super-resolution (SISR) is a notoriously challenging ill-posed problem, which aims to obtain a high-resolution (HR) output from one of its low-resolution (LR) versions. To solve the SISR problem, recently powerful deep learning algorithms have been employed and achieved the state-of-the-art performance. In this survey, we review representative deep learning-based SISR methods, and group them into two categories according to their major contributions to two essential aspects of SISR: the exploration of efficient neural network architectures for SISR, and the development of effective optimization objectives for deep SISR learning. For each category, a baseline is firstly established and several critical limitations of the baseline are summarized. Then representative works on overcoming these limitations are presented based on their original contents as well as our critical understandings and analyses, and relevant comparisons are conducted from a variety of perspectives. Finally we conclude this review with some vital current challenges and future trends in SISR leveraging deep learning algorithms.

Accepted by IEEE Transactions on Multimedia (TMM)

References in corpus (28)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Learning in Neural Networks: An Overview
• Auto-Encoding Variational Bayes
• TensorFlow: A system for large-scale machine learning
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Fully Convolutional Networks for Semantic Segmentation
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• A guide to convolution arithmetic for deep learning
• The Perception-Distortion Tradeoff
• On the Number of Linear Regions of Deep Neural Networks
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Residual Networks Behave Like Ensembles of Relatively Shallow Networks
• A note on the evaluation of generative models
• Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems
• Loss Functions for Neural Networks for Image Processing
• Deep Edge Guided Recurrent Residual Learning for Image Super-Resolution
• How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
• Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
• Is the deconvolution layer the same as a convolutional layer?
• The Shattered Gradients Problem: If resnets are the answer, then what is the question?
• Implicit Maximum Likelihood Estimation
• Dual Path Networks
• Super-Resolution via Conditional Implicit Maximum Likelihood Estimation

Cited by in corpus (96)

• Artificial Intelligence in the Creative Industries: A Review
• Remote Sensing Image Super-resolution and Object Detection: Benchmark and State of the Art
• ESRGAN+ : Further Improving Enhanced Super-Resolution Generative Adversarial Network
• Super-Resolution Analysis via Machine Learning: A Survey for Fluid Flows
• Light Field Image Super-Resolution Using Deformable Convolution
• Learned Image Downscaling for Upscaling using Content Adaptive Resampler
• HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment
• A comparative study of convolutional neural network models for wind field downscaling
• Review: Deep Learning in Electron Microscopy
• Deep Learning Methods for Solving Linear Inverse Problems: Research Directions and Paradigms
• Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision
• Super-Resolution via Image-Adapted Denoising CNNs: Incorporating External and Internal Learning
• A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution
• Transforming Image Super-Resolution: A ConvFormer-based Efficient Approach
• Learning Detail-Structure Alternative Optimization for Blind Super-Resolution
• NTIRE 2024 Challenge on Image Super-Resolution (x4): Methods and Results
• Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model
• Single-atom level determination of 3-dimensional surface atomic structure via neural network-assisted atomic electron tomography
• Partial Scanning Transmission Electron Microscopy with Deep Learning
• Deep Learning based HEp-2 Image Classification: A Comprehensive Review
• Dense Dual-Attention Network for Light Field Image Super-Resolution
• Deep Radio Interferometric Imaging with POLISH: DSA-2000 and weak lensing
• Low-field magnetic resonance image enhancement via stochastic image quality transfer
• DDistill-SR: Reparameterized Dynamic Distillation Network for Lightweight Image Super-Resolution
• Deep Learning-based Imaging in Radio Interferometry
• STDAN: Deformable Attention Network for Space-Time Video Super-Resolution
• Iterative Network for Image Super-Resolution
• Deep Learning-Based Super-Resolution and De-Noising for XMM-Newton Images
• Cephalogram Synthesis and Landmark Detection in Dental Cone-Beam CT Systems
• A Systematic Survey of Deep Learning-based Single-Image Super-Resolution
• Resolution Enhancement Processing on Low Quality Images Using Swin Transformer Based on Interval Dense Connection Strategy
• Real-World Single Image Super-Resolution: A Brief Review
• Cali-Sketch: Stroke Calibration and Completion for High-Quality Face Image Generation from Human-Like Sketches
• Increasing a microscope's effective field of view via overlapped imaging and machine learning
• Video Super-Resolution with Recurrent Structure-Detail Network
• SPI-GAN: Towards Single-Pixel Imaging through Generative Adversarial Network
• Deep Learning for Predictive Analytics in Reversible Steganography
• Learning Many-to-Many Mapping for Unpaired Real-World Image Super-resolution and Downscaling
• Neural Network Methods for Radiation Detectors and Imaging
• Instrument-To-Instrument translation: Instrumental advances drive restoration of solar observation series via deep learning
• Reconstructing Three-decade Global Fine-Grained Nighttime Light Observations by a New Super-Resolution Framework
• Video Super-resolution with Temporal Group Attention
• Adversarial Perturbations Prevail in the Y-Channel of the YCbCr Color Space
• Spatial-Angular Interaction for Light Field Image Super-Resolution
• Understanding Important Features of Deep Learning Models for Transmission Electron Microscopy Image Segmentation
• MDCN: Multi-scale Dense Cross Network for Image Super-Resolution
• Deep Learning Supersampled Scanning Transmission Electron Microscopy
• Local-Selective Feature Distillation for Single Image Super-Resolution
• Perceptual Extreme Super Resolution Network with Receptive Field Block
• Prediction of Ultrasonic Guided Wave Propagation in Solid-fluid and their Interface under Uncertainty using Machine Learning
• SeG-SR: Integrating Semantic Knowledge into Remote Sensing Image Super-Resolution via Vision-Language Model
• Deep Learning-based Face Super-Resolution: A Survey
• Image Super-Resolution via RL-CSC: When Residual Learning Meets Convolutional Sparse Coding
• SR-GAN for SR-gamma: super resolution of photon calorimeter images at collider experiments
• Reducing Spatial Discretization Error on Coarse CFD Simulations Using an OpenFOAM-Embedded Deep Learning Framework
• SplitSR: An End-to-End Approach to Super-Resolution on Mobile Devices
• Real-Time Super-Resolution System of 4K-Video Based on Deep Learning
• Cuboid-Net: A Multi-Branch Convolutional Neural Network for Joint Space-Time Video Super Resolution
• Hyperspectral Image Super-Resolution via Deep Prior Regularization with Parameter Estimation
• MIASSR: An Approach for Medical Image Arbitrary Scale Super-Resolution
• Impact of deep learning-based image super-resolution on binary signal detection
• Advancing atomic electron tomography with neural networks
• Super-Resolving Beyond Satellite Hardware Using Realistically Degraded Images
• High-Resolution Magnetic Particle Imaging System Matrix Recovery Using a Vision Transformer with Residual Feature Network
• FL-MISR: Fast Large-Scale Multi-Image Super-Resolution for Computed Tomography Based on Multi-GPU Acceleration
• Diffusion Models to Enhance the Resolution of Microscopy Images: A Tutorial
• DEff-GAN: Diverse Attribute Transfer for Few-Shot Image Synthesis
• Statistical Downscaling of Temperature Distributions from the Synoptic Scale to the Mesoscale Using Deep Convolutional Neural Networks
• Residual Channel Attention Generative Adversarial Network for Image Super-Resolution and Noise Reduction
• Simulation-based Lidar Super-resolution for Ground Vehicles
• Lightweight image super-resolution with enhanced CNN
• Scale-Aware Dynamic Network for Continuous-Scale Super-Resolution
• Dreaming up scale invariance via inverse renormalization group
• Exploring Linear Attention Alternative for Single Image Super-Resolution
• Advancing biological super-resolution microscopy through deep learning: a brief review
• Pix2Streams: Dynamic Hydrology Maps from Satellite-LiDAR Fusion
• Single Image Super-resolution with a Switch Guided Hybrid Network for Satellite Images
• Learning-Based Quality Assessment for Image Super-Resolution
• Super-resolving Compressed Images via Parallel and Series Integration of Artifact Reduction and Resolution Enhancement
• Implicit Subspace Prior Learning for Dual-Blind Face Restoration
• Disentangle Perceptual Learning through Online Contrastive Learning
• Domain Fingerprints for No-reference Image Quality Assessment
• Learned Multi-View Texture Super-Resolution
• Adaptive Loss Function for Super Resolution Neural Networks Using Convex Optimization Techniques
• Super-Resolving Commercial Satellite Imagery Using Realistic Training Data
• Pixel-Level Self-Paced Learning for Super-Resolution
• Probabilistic Super-Resolution of Solar Magnetograms: Generating Many Explanations and Measuring Uncertainties
• Optical Flow Super-Resolution Based on Image Guidence Using Convolutional Neural Network
• Single Image Super-Resolution
• Machine Vision for Improved Human-Robot Cooperation in Adverse Underwater Conditions
• Two-Stage Self-Supervised Cycle-Consistency Network for Reconstruction of Thin-Slice MR Images
• CEM-TUDASR: Computationally efficient multi-modality transformer based unsupervised domain adaptive super-resolution approach
• Time accelerated image super-resolution using shallow residual feature representative network
• Real-Time Video Super-Resolution by Joint Local Inference and Global Parameter Estimation
• Asymmetric CNN for image super-resolution
• Single image deep defocus estimation and its applications