Learning Joint 2D-3D Representations for Depth Completion
arXiv:2012.12402
Abstract
In this paper, we tackle the problem of depth completion from RGBD data. Towards this goal, we design a simple yet effective neural network block that learns to extract joint 2D and 3D features. Specifically, the block consists of two domain-specific sub-networks that apply 2D convolution on image pixels and continuous convolution on 3D points, with their output features fused in image space. We build the depth completion network simply by stacking the proposed block, which has the advantage of learning hierarchical representations that are fully fused between 2D and 3D spaces at multiple levels. We demonstrate the effectiveness of our approach on the challenging KITTI depth completion benchmark and show that our approach outperforms the state-of-the-art.
ICCV 2019
References in corpus (6)
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
- DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling
- Deep Continuous Fusion for Multi-Sensor 3D Object Detection
- Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
- Confidence Propagation through CNNs for Guided Sparse Depth Regression