Self-supervised Learning for Single View Depth and Surface Normal Estimation
arXiv:1903.00112
Abstract
In this work we present a self-supervised learning framework to simultaneously train two Convolutional Neural Networks (CNNs) to predict depth and surface normals from a single image. In contrast to most existing frameworks which represent outdoor scenes as fronto-parallel planes at piece-wise smooth depth, we propose to predict depth with surface orientation while assuming that natural scenes have piece-wise smooth normals. We show that a simple depth-normal consistency as a soft-constraint on the predictions is sufficient and effective for training both these networks simultaneously. The trained normal network provides state-of-the-art predictions while the depth network, relying on much realistic smooth normal assumption, outperforms the traditional self-supervised depth prediction network by a large margin on the KITTI benchmark. Demo video: https://youtu.be/ZD-ZRsw7hdM
6 pages, 3 figures, ICRA 2019
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
- DeMoN: Depth and Motion Network for Learning Monocular Stereo
- Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
- Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistency
- PixelNet: Towards a General Pixel-level Architecture
- Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries