3D Rigid Motion Segmentation with Mixed and Unknown Number of Models

Abstract

Many real-world video sequences cannot be conveniently categorized as general or degenerate; in such cases, imposing a false dichotomy in using the fundamental matrix or homography model for motion segmentation on video sequences would lead to difficulty. Even when we are confronted with a general scene-motion, the fundamental matrix approach as a model for motion segmentation still suffers from several defects, which we discuss in this paper. The full potential of the fundamental matrix approach could only be realized if we judiciously harness information from the simpler homography model. From these considerations, we propose a multi-model spectral clustering framework that synergistically combines multiple models (homography and fundamental matrix) together. We show that the performance can be substantially improved in this way. For general motion segmentation tasks, the number of independently moving objects is often unknown a priori and needs to be estimated from the observations. This is referred to as model selection and it is essentially still an open research problem. In this work, we propose a set of model selection criteria balancing data fidelity and model complexity. We perform extensive testing on existing motion segmentation datasets with both segmentation and model selection tasks, achieving state-of-the-art performance on all of them; we also put forth a more realistic and challenging dataset adapted from the KITTI benchmark, containing real-world effects such as strong perspectives and strong forward translations not seen in the traditional datasets.

arXiv commentIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2019. arXiv admin note: substantial text overlap with arXiv:1804.02142

References in corpus (5)

• Robust Recovery of Subspace Structures by Low-Rank Representation
• On Constrained Spectral Clustering and Its Applications
• Hybrid Linear Modeling via Local Best-fit Flats
• Background Modeling Using Adaptive Pixelwise Kernel Variances in a Hybrid Feature Space
• A Detailed Rubric for Motion Segmentation

Cited by in corpus (6)

• MultiLink: Multi-class Structure Recovery via Agglomerative Clustering and Model Selection
• Morig: Motion-aware rigging of character meshes from point clouds
• MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization
• RiWNet: A moving object instance segmentation Network being Robust in adverse Weather conditions
• U2-ONet: A Two-level Nested Octave U-structure with Multiscale Attention Mechanism for Moving Instances Segmentation
• Learning to Segment Rigid Motions from Two Frames