Self-Supervised Learning for Stereo Matching with Self-Improving Ability

Abstract

Exiting deep-learning based dense stereo matching methods often rely on ground-truth disparity maps as the training signals, which are however not always available in many situations. In this paper, we design a simple convolutional neural network architecture that is able to learn to compute dense disparity maps directly from the stereo inputs. Training is performed in an end-to-end fashion without the need of ground-truth disparity maps. The idea is to use image warping error (instead of disparity-map residuals) as the loss function to drive the learning process, aiming to find a depth-map that minimizes the warping error. While this is a simple concept well-known in stereo matching, to make it work in a deep-learning framework, many non-trivial challenges must be overcome, and in this work we provide effective solutions. Our network is self-adaptive to different unseen imageries as well as to different camera settings. Experiments on KITTI and Middlebury stereo benchmark datasets show that our method outperforms many state-of-the-art stereo matching methods with a margin, and at the same time significantly faster.

13 pages, 11 figures

References in corpus (1)

• End-to-End Learning of Geometry and Context for Deep Stereo Regression

Cited by in corpus (20)

• Hierarchical Neural Architecture Search for Deep Stereo Matching
• Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency
• A Survey on Deep Learning Architectures for Image-based Depth Reconstruction
• CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching
• Noise-Aware Unsupervised Deep Lidar-Stereo Fusion
• Displacement-Invariant Cost Computation for Efficient Stereo Matching
• Domain-invariant Stereo Matching Networks
• L3C-Stereo: Lossless Compression for Stereo Images
• Stereo Vision Based Single-Shot 6D Object Pose Estimation for Bin-Picking by a Robot Manipulator
• Efficient Depth Completion Using Learned Bases
• Expanding Sparse Guidance for Stereo Matching
• Multi-scale Cross-form Pyramid Network for Stereo Matching
• MSDC-Net: Multi-Scale Dense and Contextual Networks for Automated Disparity Map for Stereo Matching
• Achieving Domain Robustness in Stereo Matching Networks by Removing Shortcut Learning
• Single Image Deblurring and Camera Motion Estimation with Depth Map
• Deep Two-View Structure-from-Motion Revisited
• Depth Completion using Piecewise Planar Model
• Remixing Music with Visual Conditioning
• Co-Teaching: An Ark to Unsupervised Stereo Matching
• Non-destructive three-dimensional measurement of hand vein based on self-supervised network