Learning for Disparity Estimation through Feature Constancy

Abstract

Stereo matching algorithms usually consist of four steps, including matching cost calculation, matching cost aggregation, disparity calculation, and disparity refinement. Existing CNN-based methods only adopt CNN to solve parts of the four steps, or use different networks to deal with different steps, making them difficult to obtain the overall optimal solution. In this paper, we propose a network architecture to incorporate all steps of stereo matching. The network consists of three parts. The first part calculates the multi-scale shared features. The second part performs matching cost calculation, matching cost aggregation and disparity calculation to estimate the initial disparity using shared features. The initial disparity and the shared features are used to calculate the feature constancy that measures correctness of the correspondence between two input images. The initial disparity and the feature constancy are then fed to a sub-network to refine the initial disparity. The proposed method has been evaluated on the Scene Flow and KITTI datasets. It achieves the state-of-the-art performance on the KITTI 2012 and KITTI 2015 benchmarks while maintaining a very fast running time.

arXiv commentAccepted by CVPR 2018, 10 pages, 3 figures

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Highway Networks
• Residual Networks of Residual Networks: Multilevel Residual Networks
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• Self-Supervised Learning for Stereo Matching with Self-Improving Ability
• Look Wider to Match Image Patches with Convolutional Neural Networks
• Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching

Cited by in corpus (15)

• BMN: Boundary-Matching Network for Temporal Action Proposal Generation
• Pyramid Stereo Matching Network
• EdgeStereo: A Context Integrated Residual Pyramid Network for Stereo Matching
• AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching
• CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching
• Anytime Stereo Image Depth Estimation on Mobile Devices
• ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems
• SegStereo: Exploiting Semantic Information for Disparity Estimation
• On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approach
• Normal Assisted Stereo Depth Estimation
• MTStereo 2.0: improved accuracy of stereo depth estimation withMax-trees
• A Domain Agnostic Normalization Layer for Unsupervised Adversarial Domain Adaptation
• RRNet: Repetition-Reduction Network for Energy Efficient Decoder of Depth Estimation
• Confidence Inference for Focused Learning in Stereo Matching
• Shift Convolution Network for Stereo Matching