A Learning-based Framework for Hybrid Depth-from-Defocus and Stereo Matching
arXiv:1708.00583
Abstract
Depth from defocus (DfD) and stereo matching are two most studied passive depth sensing schemes. The techniques are essentially complementary: DfD can robustly handle repetitive textures that are problematic for stereo matching whereas stereo matching is insensitive to defocus blurs and can handle large depth range. In this paper, we present a unified learning-based technique to conduct hybrid DfD and stereo matching. Our input is image triplets: a stereo pair and a defocused image of one of the stereo views. We first apply depth-guided light field rendering to construct a comprehensive training dataset for such hybrid sensing setups. Next, we adopt the hourglass network architecture to separately conduct depth inference from DfD and stereo. Finally, we exploit different connection methods between the two separate networks for integrating them into a unified solution to produce high fidelity 3D disparity maps. Comprehensive experiments on real and synthetic data show that our new learning-based hybrid 3D sensing technique can significantly improve accuracy and robustness in 3D reconstruction.
References in corpus (7)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
- Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
- Computing the Stereo Matching Cost with a Convolutional Neural Network
- Stacked Hourglass Networks for Human Pose Estimation
- Look Wider to Match Image Patches with Convolutional Neural Networks
- End-to-End Training of Hybrid CNN-CRF Models for Stereo