Masked GANs for Unsupervised Depth and Pose Prediction with Scale Consistency

Abstract

Previous work has shown that adversarial learning can be used for unsupervised monocular depth and visual odometry (VO) estimation, in which the adversarial loss and the geometric image reconstruction loss are utilized as the mainly supervisory signals to train the whole unsupervised framework. However, the performance of the adversarial framework and image reconstruction is usually limited by occlusions and the visual field changes between frames. This paper proposes a masked generative adversarial network (GAN) for unsupervised monocular depth and ego-motion estimation.The MaskNet and Boolean mask scheme are designed in this framework to eliminate the effects of occlusions and impacts of visual field changes on the reconstruction loss and adversarial loss, respectively. Furthermore, we also consider the scale consistency of our pose network by utilizing a new scale-consistency loss, and therefore, our pose network is capable of providing the full camera trajectory over a long monocular sequence. Extensive experiments on the KITTI dataset show that each component proposed in this paper contributes to the performance, and both our depth and trajectory predictions achieve competitive performance on the KITTI and Make3D datasets.

arXiv comment11 pages, 7 figures

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras
• A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Monocular Depth Estimation Based On Deep Learning: An Overview
• Adversarial Examples: Opportunities and Challenges
• GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation with Generative Adversarial Networks
• Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistency
• SGANVO: Unsupervised Deep Visual Odometry and Depth Estimation with Stacked Generative Adversarial Networks
• Deep Direct Visual Odometry
• Recurrent Neural Network for (Un-)supervised Learning of Monocular VideoVisual Odometry and Depth
• Unsupervised Learning of Monocular Depth and Ego-Motion Using Multiple Masks

Cited by in corpus (9)

• MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer
• Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey
• Deep Learning-based Depth Estimation Methods from Monocular Image and Videos: A Comprehensive Survey
• Deep Direct Visual Odometry
• Self-Supervised Scale Recovery for Monocular Depth and Egomotion Estimation
• Learning Occlusion-Aware Coarse-to-Fine Depth Map for Self-supervised Monocular Depth Estimation
• MonoPP: Metric-Scaled Self-Supervised Monocular Depth Estimation by Planar-Parallax Geometry in Automotive Applications
• On Robust Cross-View Consistency in Self-Supervised Monocular Depth Estimation
• Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency