Confidence Propagation through CNNs for Guided Sparse Depth Regression

Abstract

Generally, convolutional neural networks (CNNs) process data on a regular grid, e.g. data generated by ordinary cameras. Designing CNNs for sparse and irregularly spaced input data is still an open research problem with numerous applications in autonomous driving, robotics, and surveillance. In this paper, we propose an algebraically-constrained normalized convolution layer for CNNs with highly sparse input that has a smaller number of network parameters compared to related work. We propose novel strategies for determining the confidence from the convolution operation and propagating it to consecutive layers. We also propose an objective function that simultaneously minimizes the data error while maximizing the output confidence. To integrate structural information, we also investigate fusion strategies to combine depth and RGB information in our normalized convolution network framework. In addition, we introduce the use of output confidence as an auxiliary information to improve the results. The capabilities of our normalized convolution network framework are demonstrated for the problem of scene depth completion. Comprehensive experiments are performed on the KITTI-Depth and the NYU-Depth-v2 datasets. The results clearly demonstrate that the proposed approach achieves superior performance while requiring only about 1-5% of the number of parameters compared to the state-of-the-art methods.

14 pages, 14 Figures

References in corpus (5)

• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Propagating Confidences through CNNs for Sparse Data Regression
• Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image
• In Defense of Classical Image Processing: Fast Depth Completion on the CPU
• Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation

Cited by in corpus (22)

• Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review
• Adaptive Context-Aware Multi-Modal Network for Depth Completion
• On Deep Learning Techniques to Boost Monocular Depth Estimation for Autonomous Navigation
• A Surface Geometry Model for LiDAR Depth Completion
• Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
• LiDAR-guided Stereo Matching with a Spatial Consistency Constraint
• Learning Guided Convolutional Network for Depth Completion
• Volumetric Occupancy Mapping With Probabilistic Depth Completion for Robotic Navigation
• Edge-aware Bidirectional Diffusion for Dense Depth Estimation from Light Fields
• Adaptive LiDAR Sampling and Depth Completion using Ensemble Variance
• SGTBN: Generating Dense Depth Maps from Single-Line LiDAR
• DFineNet: Ego-Motion Estimation and Depth Refinement from Sparse, Noisy Depth Input with RGB Guidance
• Deformable spatial propagation network for depth completion
• Conf-Net: Toward High-Confidence Dense 3D Point-Cloud with Error-Map Prediction
• Non-learning Stereo-aided Depth Completion under Mis-projection via Selective Stereo Matching
• RGB-D-Fusion: Image Conditioned Depth Diffusion of Humanoid Subjects
• RigNet: Repetitive Image Guided Network for Depth Completion
• DeepLiDARFlow: A Deep Learning Architecture For Scene Flow Estimation Using Monocular Camera and Sparse LiDAR
• Learning Joint 2D-3D Representations for Depth Completion
• Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency
• Depth Edge Guided CNNs for Sparse Depth Upsampling
• Balanced Depth Completion between Dense Depth Inference and Sparse Range Measurements via KISS-GP