Joint Prediction of Depths, Normals and Surface Curvature from RGB Images using CNNs

Abstract

Understanding the 3D structure of a scene is of vital importance, when it comes to developing fully autonomous robots. To this end, we present a novel deep learning based framework that estimates depth, surface normals and surface curvature by only using a single RGB image. To the best of our knowledge this is the first work to estimate surface curvature from colour using a machine learning approach. Additionally, we demonstrate that by tuning the network to infer well designed features, such as surface curvature, we can achieve improved performance at estimating depth and normals.This indicates that network guidance is still a useful aspect of designing and training a neural network. We run extensive experiments where the network is trained to infer different tasks while the model capacity is kept constant resulting in different feature maps based on the tasks at hand. We outperform the previous state-of-the-art benchmarks which jointly estimate depths and surface normals while predicting surface curvature in parallel.

References in corpus (4)

• Deep Residual Learning for Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture
• Deep Convolutional Neural Fields for Depth Estimation from a Single Image

Cited by in corpus (8)

• Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries
• ENG: End-to-end Neural Geometry for Robust Depth and Pose Estimation using CNNs
• Deep Learning based Monocular Depth Prediction: Datasets, Methods and Applications
• Pano Popups: Indoor 3D Reconstruction with a Plane-Aware Network
• Normal Assisted Stereo Depth Estimation
• Quantity beats quality for semantic segmentation of corrosion in images
• Aerial Height Prediction and Refinement Neural Networks with Semantic and Geometric Guidance
• Boundary-induced and scene-aggregated network for monocular depth prediction