Semantics-Driven Unsupervised Learning for Monocular Depth and Ego-Motion Estimation

Abstract

We propose a semantics-driven unsupervised learning approach for monocular depth and ego-motion estimation from videos in this paper. Recent unsupervised learning methods employ photometric errors between synthetic view and actual image as a supervision signal for training. In our method, we exploit semantic segmentation information to mitigate the effects of dynamic objects and occlusions in the scene, and to improve depth prediction performance by considering the correlation between depth and semantics. To avoid costly labeling process, we use noisy semantic segmentation results obtained by a pre-trained semantic segmentation network. In addition, we minimize the position error between the corresponding points of adjacent frames to utilize 3D spatial information. Experimental results on the KITTI dataset show that our method achieves good performance in both depth and ego-motion estimation tasks.

References in corpus (6)

• DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks
• DeMoN: Depth and Motion Network for Learning Monocular Stereo
• Semi-Supervised Deep Learning for Monocular Depth Map Prediction
• Recurrent Neural Network for (Un-)supervised Learning of Monocular VideoVisual Odometry and Depth
• Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding
• Beyond Photometric Loss for Self-Supervised Ego-Motion Estimation