Dense Depth Posterior (DDP) from Single Image and Sparse Range

Abstract

We present a deep learning system to infer the posterior distribution of a dense depth map associated with an image, by exploiting sparse range measurements, for instance from a lidar. While the lidar may provide a depth value for a small percentage of the pixels, we exploit regularities reflected in the training set to complete the map so as to have a probability over depth for each pixel in the image. We exploit a Conditional Prior Network, that allows associating a probability to each depth value given an image, and combine it with a likelihood term that uses the sparse measurements. Optionally we can also exploit the availability of stereo during training, but in any case only require a single image and a sparse point cloud at run-time. We test our approach on both unsupervised and supervised depth completion using the KITTI benchmark, and improve the state-of-the-art in both.

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• DeepStereo: Learning to Predict New Views from the World's Imagery
• In Defense of Classical Image Processing: Fast Depth Completion on the CPU
• High-resolution LIDAR-based Depth Mapping using Bilateral Filter

Cited by in corpus (6)

• Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving
• Learning Guided Convolutional Network for Depth Completion
• Deep RGB-D Canonical Correlation Analysis For Sparse Depth Completion
• Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR
• Depth Completion via Deep Basis Fitting
• SimuGAN: Unsupervised forward modeling and optimal design of a LIDAR Camera