Deformable Non-local Network for Video Super-Resolution

Abstract

The video super-resolution (VSR) task aims to restore a high-resolution (HR) video frame by using its corresponding low-resolution (LR) frame and multiple neighboring frames. At present, many deep learning-based VSR methods rely on optical flow to perform frame alignment. The final recovery results will be greatly affected by the accuracy of optical flow. However, optical flow estimation cannot be completely accurate, and there are always some errors. In this paper, we propose a novel deformable non-local network (DNLN) which is a non-optical-flow-based method. Specifically, we apply the deformable convolution and improve its ability of adaptive alignment at the feature level. Furthermore, we utilize a non-local structure to capture the global correlation between the reference frame and the aligned neighboring frames, and simultaneously enhance desired fine details in the aligned frames. To reconstruct the final high-quality HR video frames, we use residual in residual dense blocks to take full advantage of the hierarchical features. Experimental results on benchmark datasets demonstrate that the proposed DNLN can achieve state-of-the-art performance on VSR task.

References in corpus (5)

• Adam: A Method for Stochastic Optimization
• Video Enhancement with Task-Oriented Flow
• ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
• TDAN: Temporally Deformable Alignment Network for Video Super-Resolution
• Progressive Perception-Oriented Network for Single Image Super-Resolution

Cited by in corpus (6)

• Artificial Intelligence in the Creative Industries: A Review
• Light Field Image Super-Resolution Using Deformable Convolution
• STDAN: Deformable Attention Network for Space-Time Video Super-Resolution
• Deep Hierarchical Super Resolution for Scientific Data
• Deformable Kernel Convolutional Network for Video Extreme Super-Resolution
• Learnable Sampling 3D Convolution for Video Enhancement and Action Recognition