Monocular Dense 3D Reconstruction of a Complex Dynamic Scene from Two Perspective Frames

Abstract

This paper proposes a new approach for monocular dense 3D reconstruction of a complex dynamic scene from two perspective frames. By applying superpixel over-segmentation to the image, we model a generically dynamic (hence non-rigid) scene with a piecewise planar and rigid approximation. In this way, we reduce the dynamic reconstruction problem to a "3D jigsaw puzzle" problem which takes pieces from an unorganized "soup of superpixels". We show that our method provides an effective solution to the inherent relative scale ambiguity in structure-from-motion. Since our method does not assume a template prior, or per-object segmentation, or knowledge about the rigidity of the dynamic scene, it is applicable to a wide range of scenarios. Extensive experiments on both synthetic and real monocular sequences demonstrate the superiority of our method compared with the state-of-the-art methods.

arXiv commentInternational Conference on Computer Vision (ICCV) 2017 pp: 4649-4657

References in corpus (2)

• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• Multi-body Non-rigid Structure-from-Motion

Cited by in corpus (4)

• Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction
• Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective
• Occluded Joints Recovery in 3D Human Pose Estimation based on Distance Matrix
• Learning Single-Image Depth from Videos using Quality Assessment Networks