End-to-End Learning of Geometry and Context for Deep Stereo Regression

Abstract

We propose a novel deep learning architecture for regressing disparity from a rectified pair of stereo images. We leverage knowledge of the problem's geometry to form a cost volume using deep feature representations. We learn to incorporate contextual information using 3-D convolutions over this volume. Disparity values are regressed from the cost volume using a proposed differentiable soft argmin operation, which allows us to train our method end-to-end to sub-pixel accuracy without any additional post-processing or regularization. We evaluate our method on the Scene Flow and KITTI datasets and on KITTI we set a new state-of-the-art benchmark, while being significantly faster than competing approaches.

References in corpus (2)

• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Learning to Compare Image Patches via Convolutional Neural Networks

Cited by in corpus (13)

• Learning a Multi-View Stereo Machine
• Self-Supervised Learning for Stereo Matching with Self-Improving Ability
• Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines
• GA-Net: Guided Aggregation Net for End-to-end Stereo Matching
• Stereo R-CNN based 3D Object Detection for Autonomous Driving
• Deep Robust Single Image Depth Estimation Neural Network Using Scene Understanding
• Deep Stereo Matching with Explicit Cost Aggregation Sub-Architecture
• Learning monocular depth estimation infusing traditional stereo knowledge
• Group-wise Correlation Stereo Network
• Deep Rigid Instance Scene Flow
• Learning Independent Object Motion from Unlabelled Stereoscopic Videos
• Multi-scale Cross-form Pyramid Network for Stereo Matching
• Learning Parallax Attention for Stereo Image Super-Resolution