Self-Supervised Learning for Stereo Matching with Self-Improving Ability

Abstract

Exiting deep-learning based dense stereo matching methods often rely on ground-truth disparity maps as the training signals, which are however not always available in many situations. In this paper, we design a simple convolutional neural network architecture that is able to learn to compute dense disparity maps directly from the stereo inputs. Training is performed in an end-to-end fashion without the need of ground-truth disparity maps. The idea is to use image warping error (instead of disparity-map residuals) as the loss function to drive the learning process, aiming to find a depth-map that minimizes the warping error. While this is a simple concept well-known in stereo matching, to make it work in a deep-learning framework, many non-trivial challenges must be overcome, and in this work we provide effective solutions. Our network is self-adaptive to different unseen imageries as well as to different camera settings. Experiments on KITTI and Middlebury stereo benchmark datasets show that our method outperforms many state-of-the-art stereo matching methods with a margin, and at the same time significantly faster.

13 pages, 11 figures

References in corpus (1)

• End-to-End Learning of Geometry and Context for Deep Stereo Regression

Cited by in corpus (7)

• Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency
• A Survey on Deep Learning Architectures for Image-based Depth Reconstruction
• Noise-Aware Unsupervised Deep Lidar-Stereo Fusion
• MSDC-Net: Multi-Scale Dense and Contextual Networks for Automated Disparity Map for Stereo Matching
• Multi-scale Cross-form Pyramid Network for Stereo Matching
• Single Image Deblurring and Camera Motion Estimation with Depth Map
• Non-destructive three-dimensional measurement of hand vein based on self-supervised network