Blind Super-Resolution Kernel Estimation using an Internal-GAN
arXiv:1909.06581
Abstract
Super resolution (SR) methods typically assume that the low-resolution (LR) image was downscaled from the unknown high-resolution (HR) image by a fixed 'ideal' downscaling kernel (e.g. Bicubic downscaling). However, this is rarely the case in real LR images, in contrast to synthetically generated SR datasets. When the assumed downscaling kernel deviates from the true one, the performance of SR methods significantly deteriorates. This gave rise to Blind-SR - namely, SR when the downscaling kernel ("SR-kernel") is unknown. It was further shown that the true SR-kernel is the one that maximizes the recurrence of patches across scales of the LR image. In this paper we show how this powerful cross-scale recurrence property can be realized using Deep Internal Learning. We introduce "KernelGAN", an image-specific Internal-GAN, which trains solely on the LR test image at test time, and learns its internal distribution of patches. Its Generator is trained to produce a downscaled version of the LR test image, such that its Discriminator cannot distinguish between the patch distribution of the downscaled image, and the patch distribution of the original LR image. The Generator, once trained, constitutes the downscaling operation with the correct image-specific SR-kernel. KernelGAN is fully unsupervised, requires no training data other than the input image itself, and leads to state-of-the-art results in Blind-SR when plugged into existing SR algorithms.
Cited by in corpus (35)
- Unfolding the Alternating Optimization for Blind Super Resolution
- Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data
- Deep learning enables reference-free isotropic super-resolution for volumetric fluorescence microscopy
- Learning Detail-Structure Alternative Optimization for Blind Super-Resolution
- Cross-Quality LFW: A Database for Analyzing Cross-Resolution Image Face Recognition in Unconstrained Environments
- Toward Real-World Super-Resolution via Adaptive Downsampling Models
- Deep Blind Super-Resolution for Satellite Video
- End-to-end Alternating Optimization for Real-World Blind Super Resolution
- Blind Image Super-Resolution via Contrastive Representation Learning
- Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolution
- End-to-end Alternating Optimization for Blind Super Resolution
- Bridging Component Learning with Degradation Modelling for Blind Image Super-Resolution
- Unsupervised Degradation Representation Learning for Blind Super-Resolution
- NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Results
- Real-World Single Image Super-Resolution: A Brief Review
- Unsupervised Real-world Image Super Resolution via Domain-distance Aware Training
- Exploring Sparsity in Image Super-Resolution for Efficient Inference
- Neural Knitworks: Patched Neural Implicit Representation Networks
- The Low-Rank Simplicity Bias in Deep Networks
- From General to Specific: Online Updating for Blind Super-Resolution
- Flow-based Kernel Prior with Application to Blind Super-Resolution
- Spectrum-to-Kernel Translation for Accurate Blind Image Super-Resolution
- Unpaired Image Super-Resolution using Pseudo-Supervision
- KOALAnet: Blind Super-Resolution using Kernel-Oriented Adaptive Local Adjustment
- FreqNet: A Frequency-domain Image Super-Resolution Network with Dicrete Cosine Transform
- Bregman Proximal Framework for Deep Linear Neural Networks
- Spectral Geometric Matrix Completion
- Training Generative Adversarial Networks in One Stage
- Deep Learning for HDR Imaging: State-of-the-Art and Future Trends
- Blind Image Super-resolution with Elaborate Degradation Modeling on Noise and Kernel
- Finding Discriminative Filters for Specific Degradations in Blind Super-Resolution
- Blind Image Restoration with Flow Based Priors
- Temporal Kernel Consistency for Blind Video Super-Resolution
- MR Slice Profile Estimation by Learning to Match Internal Patch Distributions
- The Best of Both Worlds: a Framework for Combining Degradation Prediction with High Performance Super-Resolution Networks