Detail-revealing Deep Video Super-resolution

Abstract

Previous CNN-based video super-resolution approaches need to align multiple frames to the reference. In this paper, we show that proper frame alignment and motion compensation is crucial for achieving high quality results. We accordingly propose a `sub-pixel motion compensation' (SPMC) layer in a CNN framework. Analysis and experiments show the suitability of this layer in video SR. The final end-to-end, scalable CNN framework effectively incorporates the SPMC layer and fuses multiple frames to reveal image details. Our implementation can generate visually and quantitatively high-quality results, superior to current state-of-the-arts, without the need of parameter tuning.

9 pages, submitted to conference

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections
• Deep Video Deblurring
• Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness
• Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation

Cited by in corpus (14)

• Super-Resolution via Deep Learning
• TDAN: Temporally Deformable Alignment Network for Video Super-Resolution
• EDVR: Video Restoration with Enhanced Deformable Convolutional Networks
• Unified Single-Image and Video Super-Resolution via Denoising Algorithms
• Embedded deep learning in ophthalmology: Making ophthalmic imaging smarter
• Deep Back-Projection Networks for Single Image Super-resolution
• Scale-recurrent Network for Deep Image Deblurring
• A Single Video Super-Resolution GAN for Multiple Downsampling Operators based on Pseudo-Inverse Image Formation Models
• Fast Spatio-Temporal Residual Network for Video Super-Resolution
• Symmetric Parallax Attention for Stereo Image Super-Resolution
• VidFace: A Full-Transformer Solver for Video FaceHallucination with Unaligned Tiny Snapshots
• Learning for Video Super-Resolution through HR Optical Flow Estimation
• Adapting Image Super-Resolution State-of-the-arts and Learning Multi-model Ensemble for Video Super-Resolution
• Video Super-Resolution with Long-Term Self-Exemplars