Coherent Motion Segmentation in Moving Camera Videos using Optical Flow Orientations

Abstract

In moving camera videos, motion segmentation is commonly performed using the image plane motion of pixels, or optical flow. However, objects that are at different depths from the camera can exhibit different optical flows even if they share the same real-world motion. This can cause a depth-dependent segmentation of the scene. Our goal is to develop a segmentation algorithm that clusters pixels that have similar real-world motion irrespective of their depth in the scene. Our solution uses optical flow orientations instead of the complete vectors and exploits the well-known property that under camera translation, optical flow orientations are independent of object depth. We introduce a probabilistic model that automatically estimates the number of observed independent motions and results in a labeling that is consistent with real-world motion in the scene. The result of our system is that static objects are correctly identified as one segment, even if they are at different depths. Color features and information from previous frames in the video sequence are used to correct occasional errors due to the orientation-based segmentation. We present results on more than thirty videos from different benchmarks. The system is particularly robust on complex background scenes containing objects at significantly different depths

8 pages, 5 figures, in 2013 IEEE International Conference on Computer Vision (ICCV)

Cited by in corpus (5)

• The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation
• EM-driven unsupervised learning for efficient motion segmentation
• Learnable Motion Coherence for Correspondence Pruning
• Joint Unsupervised Learning of Optical Flow and Egomotion with Bi-Level Optimization
• Spherical formulation of moving object geometric constraints for monocular fisheye cameras