Super-Resolution via Image-Adapted Denoising CNNs: Incorporating External and Internal Learning

Abstract

While deep neural networks exhibit state-of-the-art results in the task of image super-resolution (SR) with a fixed known acquisition process (e.g., a bicubic downscaling kernel), they experience a huge performance loss when the real observation model mismatches the one used in training. Recently, two different techniques suggested to mitigate this deficiency, i.e., enjoy the advantages of deep learning without being restricted by the training phase. The first one follows the plug-and-play (P&P) approach that solves general inverse problems (e.g., SR) by using Gaussian denoisers for handling the prior term in model-based optimization schemes. The second builds on internal recurrence of information inside a single image, and trains a super-resolver network at test time on examples synthesized from the low-resolution image. Our work incorporates these two independent strategies, enjoying the impressive generalization capabilities of deep learning, captured by the first, and further improving it through internal learning at test time. First, we apply a recent P&P strategy to SR. Then, we show how it may become image-adaptive in test time. This technique outperforms the above two strategies on popular datasets and gives better results than other state-of-the-art methods in practical cases where the observation model is inexact or unknown in advance.

Accepted to IEEE Signal Processing Letters (extended version)

References in corpus (5)

• Deep Learning for Single Image Super-Resolution: A Brief Review
• An Online Plug-and-Play Algorithm for Regularized Image Reconstruction
• Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform
• Class-Aware Fully-Convolutional Gaussian and Poisson Denoising
• Super-Resolution via Image-Adapted Denoising CNNs: Incorporating External and Internal Learning

Cited by in corpus (7)

• Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imaging
• Super-Resolution via Image-Adapted Denoising CNNs: Incorporating External and Internal Learning
• Back-Projection based Fidelity Term for Ill-Posed Linear Inverse Problems
• DDet: Dual-path Dynamic Enhancement Network for Real-World Image Super-Resolution
• Influence of adversarial training on super-resolution turbulence reconstruction
• On the Proof of Fixed-Point Convergence for Plug-and-Play ADMM
• Self-supervised Fine-tuning for Correcting Super-Resolution Convolutional Neural Networks