Structure-Attentioned Memory Network for Monocular Depth Estimation
arXiv:1909.04594
Abstract
Monocular depth estimation is a challenging task that aims to predict a corresponding depth map from a given single RGB image. Recent deep learning models have been proposed to predict the depth from the image by learning the alignment of deep features between the RGB image and the depth domains. In this paper, we present a novel approach, named Structure-Attentioned Memory Network, to more effectively transfer domain features for monocular depth estimation by taking into account the common structure regularities (e.g., repetitive structure patterns, planar surfaces, symmetries) in domain adaptation. To this end, we introduce a new Structure-Oriented Memory (SOM) module to learn and memorize the structure-specific information between RGB image domain and the depth domain. More specifically, in the SOM module, we develop a Memorable Bank of Filters (MBF) unit to learn a set of filters that memorize the structure-aware image-depth residual pattern, and also an Attention Guided Controller (AGC) unit to control the filter selection in the MBF given image features queries. Given the query image feature, the trained SOM module is able to adaptively select the best customized filters for cross-domain feature transferring with an optimal structural disparity between image and depth. In summary, we focus on addressing this structure-specific domain adaption challenge by proposing a novel end-to-end multi-scale memorable network for monocular depth estimation. The experiments show that our proposed model demonstrates the superior performance compared to the existing supervised monocular depth estimation approaches on the challenging KITTI and NYU Depth V2 benchmarks.
arXiv comment8 pages, 6 figures
References in corpus (14)
- Learning Transferable Features with Deep Adaptation Networks
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
- Adversarial Discriminative Domain Adaptation
- Deep CORAL: Correlation Alignment for Deep Domain Adaptation
- Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
- Depth Extraction from Video Using Non-parametric Sampling
- Unsupervised Learning of Depth and Ego-Motion from Video
- Deep Ordinal Regression Network for Monocular Depth Estimation
- Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction
- PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing
- Domain Generalization for Object Recognition with Multi-task Autoencoders
- 3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Image
- Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss