DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling

Abstract

We describe a technique that automatically generates plausible depth maps from videos using non-parametric depth sampling. We demonstrate our technique in cases where past methods fail (non-translating cameras and dynamic scenes). Our technique is applicable to single images as well as videos. For videos, we use local motion cues to improve the inferred depth maps, while optical flow is used to ensure temporal depth consistency. For training and evaluation, we use a Kinect-based system to collect a large dataset containing stereoscopic videos with known depths. We show that our depth estimation technique outperforms the state-of-the-art on benchmark databases. Our technique can be used to automatically convert a monoscopic video into stereo for 3D visualization, and we demonstrate this through a variety of visually pleasing results for indoor and outdoor scenes, including results from the feature film Charade.

Cited by in corpus (13)

• DeepStereo: Learning to Predict New Views from the World's Imagery
• Deep Robust Single Image Depth Estimation Neural Network Using Scene Understanding
• Depth Reconstruction and Computer-Aided Polyp Detection in Optical Colonoscopy Video Frames
• Moving Indoor: Unsupervised Video Depth Learning in Challenging Environments
• Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation
• Unsupervised High-Resolution Depth Learning From Videos With Dual Networks
• Semantic Hierarchical Priors for Intrinsic Image Decomposition
• FusionMapping: Learning Depth Prediction with Monocular Images and 2D Laser Scans
• Inverse Rendering Techniques for Physically Grounded Image Editing
• Depth Map Estimation of Dynamic Scenes Using Prior Depth Information
• Deep Classification Network for Monocular Depth Estimation
• What Do Single-view 3D Reconstruction Networks Learn?
• Learning to Reconstruct and Understand Indoor Scenes from Sparse Views