Robust Consistent Video Depth Estimation

Abstract

We present an algorithm for estimating consistent dense depth maps and camera poses from a monocular video. We integrate a learning-based depth prior, in the form of a convolutional neural network trained for single-image depth estimation, with geometric optimization, to estimate a smooth camera trajectory as well as detailed and stable depth reconstruction. Our algorithm combines two complementary techniques: (1) flexible deformation-splines for low-frequency large-scale alignment and (2) geometry-aware depth filtering for high-frequency alignment of fine depth details. In contrast to prior approaches, our method does not require camera poses as input and achieves robust reconstruction for challenging hand-held cell phone captures containing a significant amount of noise, shake, motion blur, and rolling shutter deformations. Our method quantitatively outperforms state-of-the-arts on the Sintel benchmark for both depth and pose estimations and attains favorable qualitative results across diverse wild datasets.

arXiv commentProject website: https://robust-cvd.github.io/

References in corpus (10)

• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• DeMoN: Depth and Motion Network for Learning Monocular Stereo
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling
• SfM-Net: Learning of Structure and Motion from Video
• DeepV2D: Video to Depth with Differentiable Structure from Motion
• DPSNet: End-to-end Deep Plane Sweep Stereo
• Self-supervised Object Motion and Depth Estimation from Video