Depth Extraction from Video Using Non-parametric Sampling

Abstract

We describe a technique that automatically generates plausible depth maps from videos using non-parametric depth sampling. We demonstrate our technique in cases where past methods fail (non-translating cameras and dynamic scenes). Our technique is applicable to single images as well as videos. For videos, we use local motion cues to improve the inferred depth maps, while optical flow is used to ensure temporal depth consistency. For training and evaluation, we use a Kinect-based system to collect a large dataset containing stereoscopic videos with known depths. We show that our depth estimation technique outperforms the state-of-the-art on benchmark databases. Our technique can be used to automatically convert a monoscopic video into stereo for 3D visualization, and we demonstrate this through a variety of visually pleasing results for indoor and outdoor scenes, including results from the feature film Charade.

arXiv admin note: text overlap with arXiv:2001.00987

References in corpus (1)

• Towards Holistic Scene Understanding: Feedback Enabled Cascaded Classification Models

Cited by in corpus (32)

• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Learning Depth from Single Images with Deep Neural Network Embedding Focal Length
• Detail Preserving Depth Estimation from a Single Image Using Attention Guided Networks
• Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image
• Semi-Supervised Deep Learning for Monocular Depth Map Prediction
• Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving
• GeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation
• MegaDepth: Learning Single-View Depth Prediction from Internet Photos
• MiniNet: An extremely lightweight convolutional neural network for real-time unsupervised monocular depth estimation
• Predicting Complete 3D Models of Indoor Scenes
• Dense Correspondences Across Scenes and Scales
• Absolute distance prediction based on deep learning object detection and monocular depth estimation models
• Depth from a Single Image by Harmonizing Overcomplete Local Network Predictions
• HANDS: A Multimodal Dataset for Modeling Towards Human Grasp Intent Inference in Prosthetic Hands
• DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image
• Single View Stereo Matching
• NVS-MonoDepth: Improving Monocular Depth Prediction with Novel View Synthesis
• T2Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Tasks
• An Approximate Shading Model with Detail Decomposition for Object Relighting
• Structure-Attentioned Memory Network for Monocular Depth Estimation
• Structured Depth Prediction in Challenging Monocular Video Sequences
• Partially Supervised Multi-Task Network for Single-View Dietary Assessment
• Coupled Depth Learning
• Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation
• Complete 3D Scene Parsing from an RGBD Image
• Improving Depth Estimation using Location Information
• Inverse Rendering Techniques for Physically Grounded Image Editing
• Temporal shape super-resolution by intra-frame motion encoding using high-fps structured light
• Synthesizing Photorealistic Images with Deep Generative Learning
• Automatic Scene Inference for 3D Object Compositing
• Contrastive Feature Loss for Image Prediction