Unified Single-Image and Video Super-Resolution via Denoising Algorithms

Abstract

Single Image Super-Resolution (SISR) aims to recover a high-resolution image from a given low-resolution version of it. Video Super Resolution (VSR) targets series of given images, aiming to fuse them to create a higher resolution outcome. Although SISR and VSR seem to have a lot in common, most SISR algorithms do not have a simple and direct extension to VSR. VSR is considered a more challenging inverse problem, mainly due to its reliance on a sub-pixel accurate motion-estimation, which has no parallel in SISR. Another complication is the dynamics of the video, often addressed by simply generating a single frame instead of a complete output sequence. In this work we suggest a simple and robust super-resolution framework that can be applied to single images and easily extended to video. Our work relies on the observation that denoising of images and videos is well-managed and very effectively treated by a variety of methods. We exploit the Plug-and-Play-Prior framework and the Regularization-by-Denoising (RED) approach that extends it, and show how to use such denoisers in order to handle the SISR and the VSR problems using a unified formulation and framework. This way, we benefit from the effectiveness and efficiency of existing image/video denoising algorithms, while solving much more challenging problems. More specifically, harnessing the VBM3D video denoiser, we obtain a strongly competitive motion-estimation free VSR algorithm, showing tendency to a high-quality output and fast processing.

References in corpus (7)

• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
• FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising
• Image Deblurring and Super-resolution by Adaptive Sparse Domain Selection and Adaptive Regularization
• Denoising Prior Driven Deep Neural Network for Image Restoration
• Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation
• Learning Deep CNN Denoiser Prior for Image Restoration
• Detail-revealing Deep Video Super-resolution

Cited by in corpus (4)

• Dynamic 3-D measurement based on fringe-to-fringe transformation using deep learning
• TMP: Temporal Motion Propagation for Online Video Super-Resolution
• TFPnP: Tuning-free Plug-and-Play Proximal Algorithm with Applications to Inverse Imaging Problems
• When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey