DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling
arXiv:2001.00987 · doi:10.1109/TPAMI.2014.2316835
Abstract
We describe a technique that automatically generates plausible depth maps from videos using non-parametric depth sampling. We demonstrate our technique in cases where past methods fail (non-translating cameras and dynamic scenes). Our technique is applicable to single images as well as videos. For videos, we use local motion cues to improve the inferred depth maps, while optical flow is used to ensure temporal depth consistency. For training and evaluation, we use a Kinect-based system to collect a large dataset containing stereoscopic videos with known depths. We show that our depth estimation technique outperforms the state-of-the-art on benchmark databases. Our technique can be used to automatically convert a monoscopic video into stereo for 3D visualization, and we demonstrate this through a variety of visually pleasing results for indoor and outdoor scenes, including results from the feature film Charade.
References in corpus (1)
Cited by in corpus (69)
- Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
- Light Field Video Capture Using a Learning-Based Hybrid Imaging System
- Single Image Depth Estimation: An Overview
- A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence
- Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image
- Single-Image Depth Perception in the Wild
- DeepStereo: Learning to Predict New Views from the World's Imagery
- Deep Learning-based Depth Estimation Methods from Monocular Image and Videos: A Comprehensive Survey
- PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing
- On Deep Learning Techniques to Boost Monocular Depth Estimation for Autonomous Navigation
- Enforcing geometric constraints of virtual normal for depth prediction
- Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networks
- Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation
- Adversarial Patch Attacks on Monocular Depth Estimation Networks
- Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimation
- Monocular Object Instance Segmentation and Depth Ordering with CNNs
- Unsupervised Monocular Depth Estimation with Left-Right Consistency
- Monocular Depth Estimation using Multi-Scale Continuous CRFs as Sequential Deep Networks
- Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding
- Joint Self-supervised Depth and Optical Flow Estimation towards Dynamic Objects
- Deep Robust Single Image Depth Estimation Neural Network Using Scene Understanding
- Less is More: Consistent Video Depth Estimation with Masked Frames Modeling
- Depth Reconstruction and Computer-Aided Polyp Detection in Optical Colonoscopy Video Frames
- DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image
- Pyramid Frequency Network with Spatial Attention Residual Refinement Module for Monocular Depth Estimation
- Deep Convolutional Neural Fields for Depth Estimation from a Single Image
- Exploiting temporal consistency for real-time video depth estimation
- Depth estimation from 4D light field videos
- SharinGAN: Combining Synthetic and Real Data for Unsupervised Geometry Estimation
- Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation
- VisualEchoes: Spatial Image Representation Learning through Echolocation
- Moving Indoor: Unsupervised Video Depth Learning in Challenging Environments
- Unsupervised High-Resolution Depth Learning From Videos With Dual Networks
- A Compromise Principle in Deep Monocular Depth Estimation
- Consistent Video Depth Estimation
- 3dDepthNet: Point Cloud Guided Depth Completion Network for Sparse Depth and Single Color Image
- Towards Real-Time Monocular Depth Estimation for Robotics: A Survey
- Monocular Depth Estimation with Augmented Ordinal Depth Relationships
- Learning Depth from Monocular Videos Using Synthetic Data: A Temporally-Consistent Domain Adaptation Approach
- TW-SMNet: Deep Multitask Learning of Tele-Wide Stereo Matching
- Learning Gaze Transitions from Depth to Improve Video Saliency Estimation
- DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency
- Unsupervised Video Depth Estimation Based on Ego-motion and Disparity Consensus
- Temporal Tessellation: A Unified Approach for Video Analysis
- Generating and Exploiting Probabilistic Monocular Depth Estimates
- Structured Depth Prediction in Challenging Monocular Video Sequences
- Learning Joint 2D-3D Representations for Depth Completion
- Semantic Hierarchical Priors for Intrinsic Image Decomposition
- Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction
- Efficient and Scalable View Generation from a Single Image using Fully Convolutional Networks
- Coupled Depth Learning
- Self-Supervised Learning of Depth and Motion Under Photometric Inconsistency
- Robust Consistent Video Depth Estimation
- Shape and Symmetry Induction for 3D Objects
- Improving Monocular Depth Estimation by Leveraging Structural Awareness and Complementary Datasets
- MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments
- FusionMapping: Learning Depth Prediction with Monocular Images and 2D Laser Scans
- Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervision
- Unsupervised Monocular Depth Perception: Focusing on Moving Objects
- Adversarial View-Consistent Learning for Monocular Depth Estimation
- Deep Classification Network for Monocular Depth Estimation
- Inverse Rendering Techniques for Physically Grounded Image Editing
- Depth Map Estimation of Dynamic Scenes Using Prior Depth Information
- Adversarial Structure Matching for Structured Prediction Tasks
- Boundary-induced and scene-aggregated network for monocular depth prediction
- Temporally Consistent Depth Prediction with Flow-Guided Memory Units
- What Do Single-view 3D Reconstruction Networks Learn?
- On the Synergies between Machine Learning and Binocular Stereo for Depth Estimation from Images: a Survey
- Learning to Reconstruct and Understand Indoor Scenes from Sparse Views