Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness

Abstract

Recently, convolutional networks (convnets) have proven useful for predicting optical flow. Much of this success is predicated on the availability of large datasets that require expensive and involved data acquisition and laborious la- beling. To bypass these challenges, we propose an unsuper- vised approach (i.e., without leveraging groundtruth flow) to train a convnet end-to-end for predicting optical flow be- tween two images. We use a loss function that combines a data term that measures photometric constancy over time with a spatial term that models the expected variation of flow across the image. Together these losses form a proxy measure for losses based on the groundtruth flow. Empiri- cally, we show that a strong convnet baseline trained with the proposed unsupervised approach outperforms the same network trained with supervision on the KITTI dataset.

References in corpus (3)

• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• Spatio-temporal video autoencoder with differentiable memory

Cited by in corpus (10)

• A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration
• Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge
• Detail-revealing Deep Video Super-resolution
• Guided Optical Flow Learning
• Optical Flow Estimation using a Spatial Pyramid Network
• Hybrid Learning of Optical Flow and Next Frame Prediction to Boost Optical Flow in the Wild
• TransFlow: Unsupervised Motion Flow by Joint Geometric and Pixel-level Estimation
• Flow2Stereo: Effective Self-Supervised Learning of Optical Flow and Stereo Matching
• Revisiting Optical Flow Estimation in 360 Videos
• Large-Scale Mapping of Human Activity using Geo-Tagged Videos