Unfolding the Alternating Optimization for Blind Super Resolution

Abstract

Previous methods decompose blind super resolution (SR) problem into two sequential steps: \textit{i}) estimating blur kernel from given low-resolution (LR) image and \textit{ii}) restoring SR image based on estimated kernel. This two-step solution involves two independently trained models, which may not be well compatible with each other. Small estimation error of the first step could cause severe performance drop of the second one. While on the other hand, the first step can only utilize limited information from LR image, which makes it difficult to predict highly accurate blur kernel. Towards these issues, instead of considering these two steps separately, we adopt an alternating optimization algorithm, which can estimate blur kernel and restore SR image in a single model. Specifically, we design two convolutional neural modules, namely \textit{Restorer} and \textit{Estimator}. \textit{Restorer} restores SR image based on predicted kernel, and \textit{Estimator} estimates blur kernel with the help of restored SR image. We alternate these two modules repeatedly and unfold this process to form an end-to-end trainable network. In this way, \textit{Estimator} utilizes information from both LR and SR images, which makes the estimation of blur kernel easier. More importantly, \textit{Restorer} is trained with the kernel estimated by \textit{Estimator}, instead of ground-truth kernel, thus \textit{Restorer} could be more tolerant to the estimation error of \textit{Estimator}. Extensive experiments on synthetic datasets and real-world images show that our model can largely outperform state-of-the-art methods and produce more visually favorable results at much higher speed. The source code is available at https://github.com/greatlog/DAN.git.

Conference on Neural Information Processing Systems (NeurIPS) 2020

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Sketch-based Manga Retrieval using Manga109 Dataset
• ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
• Wide Activation for Efficient and Accurate Image Super-Resolution
• Blind Super-Resolution Kernel Estimation using an Internal-GAN
• Unified Dynamic Convolutional Network for Super-Resolution with Variational Degradations

Cited by in corpus (9)

• Blind Super-Resolution via Meta-learning and Markov Chain Monte Carlo Simulation
• End-to-end Alternating Optimization for Real-World Blind Super Resolution
• A Systematic Survey of Deep Learning-based Single-Image Super-Resolution
• Bridging Component Learning with Degradation Modelling for Blind Image Super-Resolution
• Joint Learning Content and Degradation Aware Feature for Blind Super-Resolution
• A High-Performance Accelerator for Super-Resolution Processing on Embedded GPU
• Approaching the Limit of Image Rescaling via Flow Guidance
• Finding Discriminative Filters for Specific Degradations in Blind Super-Resolution
• The Best of Both Worlds: a Framework for Combining Degradation Prediction with High Performance Super-Resolution Networks