Depth from a Single Image by Harmonizing Overcomplete Local Network Predictions

Abstract

A single color image can contain many cues informative towards different aspects of local geometric structure. We approach the problem of monocular depth estimation by using a neural network to produce a mid-level representation that summarizes these cues. This network is trained to characterize local scene geometry by predicting, at every image location, depth derivatives of different orders, orientations and scales. However, instead of a single estimate for each derivative, the network outputs probability distributions that allow it to express confidence about some coefficients, and ambiguity about others. Scene depth is then estimated by harmonizing this overcomplete set of network predictions, using a globalization procedure that finds a single consistent depth map that best matches all the local derivative distributions. We demonstrate the efficacy of this approach through evaluation on the NYU v2 depth data set.

NIPS 2016. Project page at http://www.ttic.edu/chakrabarti/mdepth/

References in corpus (3)

• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• Depth Extraction from Video Using Non-parametric Sampling

Cited by in corpus (9)

• Unsupervised Scale-consistent Depth Learning from Video
• Peeking Behind Objects: Layered Depth Prediction from a Single Image
• Deep Depth Completion of a Single RGB-D Image
• Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries
• PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image
• Critical Contours: An Invariant Linking Image Flow with Salient Surface Organization
• Adversarial View-Consistent Learning for Monocular Depth Estimation
• Shape from Shading through Shape Evolution
• Relative Depth Order Estimation Using Multi-scale Densely Connected Convolutional Networks