Guided Optical Flow Learning

Abstract

We study the unsupervised learning of CNNs for optical flow estimation using proxy ground truth data. Supervised CNNs, due to their immense learning capacity, have shown superior performance on a range of computer vision problems including optical flow prediction. They however require the ground truth flow which is usually not accessible except on limited synthetic data. Without the guidance of ground truth optical flow, unsupervised CNNs often perform worse as they are naturally ill-conditioned. We therefore propose a novel framework in which proxy ground truth data generated from classical approaches is used to guide the CNN learning. The models are further refined in an unsupervised fashion using an image reconstruction loss. Our guided learning approach is competitive with or superior to state-of-the-art approaches on three standard benchmark datasets yet is completely unsupervised and can run in real time.

CVPR17 Workshop. Code available at https://github.com/bryanyzhu/GuidedNet

References in corpus (6)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• FlowNet: Learning Optical Flow with Convolutional Networks
• Hidden Two-Stream Convolutional Networks for Action Recognition
• Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness
• Optical Flow Estimation using a Spatial Pyramid Network
• Depth2Action: Exploring Embedded Depth for Large-Scale Action Recognition

Cited by in corpus (10)

• EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
• Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge
• Hidden Two-Stream Convolutional Networks for Action Recognition
• Lucid Data Dreaming for Video Object Segmentation
• Towards Universal Representation for Unseen Action Recognition
• NeurReg: Neural Registration and Its Application to Image Segmentation
• Deep Optical Flow Estimation Via Multi-Scale Correspondence Structure Learning
• Optical Flow Super-Resolution Based on Image Guidence Using Convolutional Neural Network
• Exploring Temporal Information for Improved Video Understanding
• Large-Scale Mapping of Human Activity using Geo-Tagged Videos