Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistency

Abstract

Learning to reconstruct depths in a single image by watching unlabeled videos via deep convolutional network (DCN) is attracting significant attention in recent years. In this paper, we introduce a surface normal representation for unsupervised depth estimation framework. Our estimated depths are constrained to be compatible with predicted normals, yielding more robust geometry results. Specifically, we formulate an edge-aware depth-normal consistency term, and solve it by constructing a depth-to-normal layer and a normal-to-depth layer inside of the DCN. The depth-to-normal layer takes estimated depths as input, and computes normal directions using cross production based on neighboring pixels. Then given the estimated normals, the normal-to-depth layer outputs a regularized depth map through local planar smoothness. Both layers are computed with awareness of edges inside the image to help address the issue of depth/normal discontinuity and preserve sharp edges. Finally, to train the network, we apply the photometric error and gradient smoothness for both depth and normal predictions. We conducted experiments on both outdoor (KITTI) and indoor (NYUv2) datasets, and show that our algorithm vastly outperforms state of the art, which demonstrates the benefits from our approach.

Accepted at AAAI 2018

References in corpus (4)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Unsupervised Learning of Depth and Ego-Motion from Video
• Semi-Supervised Deep Learning for Monocular Depth Map Prediction
• Pyramidal Gradient Matching for Optical Flow Estimation

Cited by in corpus (31)

• Monocular Depth Estimation Based On Deep Learning: An Overview
• Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey
• Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video
• Masked GANs for Unsupervised Depth and Pose Prediction with Scale Consistency
• On Deep Learning Techniques to Boost Monocular Depth Estimation for Autonomous Navigation
• Self-Supervised Monocular Depth Estimation with Self-Reference Distillation and Disparity Offset Refinement
• Unsupervised Monocular Depth Learning in Dynamic Scenes
• Semantically-Guided Representation Learning for Self-Supervised Monocular Depth
• Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud
• LEGO: Learning Edge with Geometry all at Once by Watching Videos
• Monocular Depth Estimation with Self-supervised Instance Adaptation
• Feature-metric Loss for Self-supervised Learning of Depth and Egomotion
• Joint Unsupervised Learning of Optical Flow and Depth by Watching Stereo Videos
• The Edge of Depth: Explicit Constraints between Segmentation and Depth
• VisualEchoes: Spatial Image Representation Learning through Echolocation
• Self-supervised Learning for Single View Depth and Surface Normal Estimation
• Semantic-Guided Representation Enhancement for Self-supervised Monocular Trained Depth Estimation
• Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark
• Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications
• DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency
• Calibrating Self-supervised Monocular Depth Estimation
• Instance-wise Depth and Motion Learning from Monocular Videos
• Radar-Camera Pixel Depth Association for Depth Completion
• GroundNet: Monocular Ground Plane Normal Estimation with Geometric Consistency
• Variational Monocular Depth Estimation for Reliability Prediction
• Improving Depth Estimation using Location Information
• Self-Guided Instance-Aware Network for Depth Completion and Enhancement
• LO-Net: Deep Real-time Lidar Odometry
• A Dark Flash Normal Camera
• PLG-IN: Pluggable Geometric Consistency Loss with Wasserstein Distance in Monocular Depth Estimation
• Residual-Guided Learning Representation for Self-Supervised Monocular Depth Estimation