Adversarial View-Consistent Learning for Monocular Depth Estimation

Abstract

This paper addresses the problem of Monocular Depth Estimation (MDE). Existing approaches on MDE usually model it as a pixel-level regression problem, ignoring the underlying geometry property. We empirically find this may result in sub-optimal solution: while the predicted depth map presents small loss value in one specific view, it may exhibit large loss if viewed in different directions. In this paper, inspired by multi-view stereo (MVS), we propose an Adversarial View-Consistent Learning (AVCL) framework to force the estimated depth map to be all reasonable viewed from multiple views. To this end, we first design a differentiable depth map warping operation, which is end-to-end trainable, and then propose a pose generator to generate novel views for a given image in an adversarial manner. Collaborating with the differentiable depth map warping operation, the pose generator encourages the depth estimation network to learn from hard views, hence produce view-consistent depth maps . We evaluate our method on NYU Depth V2 dataset and the experimental results show promising performance gain upon state-of-the-art MDE approaches.

arXiv commentBMVC 2019 Spotlight

References in corpus (8)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling
• Unsupervised Learning of Depth and Ego-Motion from Video
• Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis
• Depth from a Single Image by Harmonizing Overcomplete Local Network Predictions