Normal Assisted Stereo Depth Estimation

Abstract

Accurate stereo depth estimation plays a critical role in various 3D tasks in both indoor and outdoor environments. Recently, learning-based multi-view stereo methods have demonstrated competitive performance with a limited number of views. However, in challenging scenarios, especially when building cross-view correspondences is hard, these methods still cannot produce satisfying results. In this paper, we study how to leverage a normal estimation model and the predicted normal maps to improve the depth quality. We couple the learning of a multi-view normal estimation module and a multi-view depth estimation module. In addition, we propose a novel consistency loss to train an independent consistency module that refines the depths from depth/normal pairs. We find that the joint learning can improve both the prediction of normal and depth, and the accuracy & smoothness can be further improved by enforcing the consistency. Experiments on MVS, SUN3D, RGBD, and Scenes11 demonstrate the effectiveness of our method and state-of-the-art performance.

References in corpus (14)

• DeMoN: Depth and Motion Network for Learning Monocular Stereo
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Indoor Semantic Segmentation using depth information
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture
• Pyramid Stereo Matching Network
• Enforcing geometric constraints of virtual normal for depth prediction
• DeepMVS: Learning Multi-view Stereopsis
• Learning for Disparity Estimation through Feature Constancy
• Designing Deep Networks for Surface Normal Estimation
• Deep Depth Completion of a Single RGB-D Image
• Joint Prediction of Depths, Normals and Surface Curvature from RGB Images using CNNs
• Self-supervised Learning for Single View Depth and Surface Normal Estimation
• Deep Surface Normal Estimation with Hierarchical RGB-D Fusion

Cited by in corpus (1)

• Consistent Video Depth Estimation