Learning Depth from Single Images with Deep Neural Network Embedding Focal Length

Abstract

Learning depth from a single image, as an important issue in scene understanding, has attracted a lot of attention in the past decade. The accuracy of the depth estimation has been improved from conditional Markov random fields, non-parametric methods, to deep convolutional neural networks most recently. However, there exist inherent ambiguities in recovering 3D from a single 2D image. In this paper, we first prove the ambiguity between the focal length and monocular depth learning, and verify the result using experiments, showing that the focal length has a great influence on accurate depth recovery. In order to learn monocular depth by embedding the focal length, we propose a method to generate synthetic varying-focal-length dataset from fixed-focal-length datasets, and a simple and effective method is implemented to fill the holes in the newly generated images. For the sake of accurate depth recovery, we propose a novel deep neural network to infer depth through effectively fusing the middle-level information on the fixed-focal-length dataset, which outperforms the state-of-the-art methods built on pre-trained VGG. Furthermore, the newly generated varying-focal-length dataset is taken as input to the proposed network in both learning and inference phases. Extensive experiments on the fixed- and varying-focal-length datasets demonstrate that the learned monocular depth with embedded focal length is significantly improved compared to that without embedding the focal length information.

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Training Very Deep Networks
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• Richer Convolutional Features for Edge Detection
• Learning Deconvolution Network for Semantic Segmentation
• Deep multi-scale video prediction beyond mean square error
• Holistically-Nested Edge Detection
• Depth Extraction from Video Using Non-parametric Sampling
• Exploiting Depth from Single Monocular Images for Object Detection and Semantic Segmentation
• RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation
• The BerHu penalty and the grouped effect

Cited by in corpus (33)

• DeepSTORM3D: dense three dimensional localization microscopy and point spread function design by deep learning
• Adaptive Context-Aware Multi-Modal Network for Depth Completion
• Approaches, Challenges, and Applications for Deep Visual Odometry: Toward to Complicated and Emerging Areas
• A Comparative Study on Polyp Classification using Convolutional Neural Networks
• Boosting Occluded Image Classification via Subspace Decomposition Based Estimation of Deep Features
• Multi-Camera Collaborative Depth Prediction via Consistent Structure Estimation
• A Geometric Approach to Obtain a Bird's Eye View from an Image
• Deep Optics for Monocular Depth Estimation and 3D Object Detection
• Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR
• Deep Learning Network Based Spectrum Sensing Methods for OFDM Systems
• SharinGAN: Combining Synthetic and Real Data for Unsupervised Geometry Estimation
• Double Refinement Network for Efficient Indoor Monocular Depth Estimation
• Real-time Prediction of Automotive Collision Risk from Monocular Video
• Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation
• Deep Learning based Monocular Depth Prediction: Datasets, Methods and Applications
• MDFN: Multi-Scale Deep Feature Learning Network for Object Detection
• Plug-and-Play Rescaling Based Crowd Counting in Static Images
• CAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth
• Adversarially Approximated Autoencoder for Image Generation and Manipulation
• Learning Depth from Monocular Videos Using Synthetic Data: A Temporally-Consistent Domain Adaptation Approach
• SGNet: A Super-class Guided Network for Image Classification and Object Detection
• Unsupervised Deep Feature Transfer for Low Resolution Image Classification
• Automatically Extract the Semi-transparent Motion-blurred Hand from a Single Image
• MDCN: Multi-Scale, Deep Inception Convolutional Neural Networks for Efficient Object Detection
• Self-Orthogonality Module: A Network Architecture Plug-in for Learning Orthogonal Filters
• ZoomCount: A Zooming Mechanism for Crowd Counting in Static Images
• Adaptively Denoising Proposal Collection for Weakly Supervised Object Localization
• $S^3$Net: Semantic-Aware Self-supervised Depth Estimation with Monocular Videos and Synthetic Data
• Location-Aware Box Reasoning for Anchor-Based Single-Shot Object Detection
• End-to-End Framework for Efficient Deep Learning Using Metasurfaces Optics
• Camera Pose Matters: Improving Depth Prediction by Mitigating Pose Distribution Bias
• Toward Learning a Unified Many-to-Many Mapping for Diverse Image Translation
• Stacked Wasserstein Autoencoder