Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation
arXiv:1808.00769
Abstract
Convolutional neural networks are designed for dense data, but vision data is often sparse (stereo depth, point clouds, pen stroke, etc.). We present a method to handle sparse depth data with optional dense RGB, and accomplish depth completion and semantic segmentation changing only the last layer. Our proposal efficiently learns sparse features without the need of an additional validity mask. We show how to ensure network robustness to varying input sparsities. Our method even works with densities as low as 0.8% (8 layer lidar), and outperforms all published state-of-the-art on the Kitti depth completion benchmark.
3DV 2018
References in corpus (8)
- Improved Regularization of Convolutional Neural Networks with Cutout
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
- Semi-Supervised Deep Learning for Monocular Depth Map Prediction
- In Defense of Classical Image Processing: Fast Depth Completion on the CPU
- Deep Depth Completion of a Single RGB-D Image
- SBNet: Sparse Blocks Network for Fast Inference
Cited by in corpus (6)
- Confidence Propagation through CNNs for Guided Sparse Depth Regression
- Learning Guided Convolutional Network for Depth Completion
- Plug-and-Play: Improve Depth Estimation via Sparse Data Propagation
- Efficient Depth Completion Using Learned Bases
- Depth Completion via Deep Basis Fitting
- Depth Completion using Piecewise Planar Model