Revisiting Adaptive Convolutions for Video Frame Interpolation

Abstract

Video frame interpolation, the synthesis of novel views in time, is an increasingly popular research direction with many new papers further advancing the state of the art. But as each new method comes with a host of variables that affect the interpolation quality, it can be hard to tell what is actually important for this task. In this work, we show, somewhat surprisingly, that it is possible to achieve near state-of-the-art results with an older, simpler approach, namely adaptive separable convolutions, by a subtle set of low level improvements. In doing so, we propose a number of intuitive but effective techniques to improve the frame interpolation quality, which also have the potential to other related applications of adaptive convolutions such as burst image denoising, joint image filtering, or video prediction.

arXiv commentWACV 2021, http://sniklaus.com/resepconv

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Learning-Based View Synthesis for Light Field Cameras
• Making Convolutional Networks Shift-Invariant Again
• Quadratic video interpolation
• Deep Slow Motion Video Reconstruction with Hybrid Imaging System
• Multi-Kernel Prediction Networks for Denoising of Burst Images
• Frame Interpolation with Multi-Scale Deep Loss Functions and Generative Adversarial Networks
• MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement
• Spatio-Temporal Filter Adaptive Network for Video Deblurring
• Delving Deeper into Anti-aliasing in ConvNets
• Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolution
• Enhanced Quadratic Video Interpolation