End-to-End Learning of Geometry and Context for Deep Stereo Regression

Abstract

We propose a novel deep learning architecture for regressing disparity from a rectified pair of stereo images. We leverage knowledge of the problem's geometry to form a cost volume using deep feature representations. We learn to incorporate contextual information using 3-D convolutions over this volume. Disparity values are regressed from the cost volume using a proposed differentiable soft argmin operation, which allows us to train our method end-to-end to sub-pixel accuracy without any additional post-processing or regularization. We evaluate our method on the Scene Flow and KITTI datasets and on KITTI we set a new state-of-the-art benchmark, while being significantly faster than competing approaches.

References in corpus (2)

• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Learning to Compare Image Patches via Convolutional Neural Networks

Cited by in corpus (35)

• Learning a Multi-View Stereo Machine
• Self-Supervised Learning for Stereo Matching with Self-Improving Ability
• Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines
• GA-Net: Guided Aggregation Net for End-to-end Stereo Matching
• Stereo R-CNN based 3D Object Detection for Autonomous Driving
• AANet: Adaptive Aggregation Network for Efficient Stereo Matching
• Deep Robust Single Image Depth Estimation Neural Network Using Scene Understanding
• Deep Stereo Matching with Explicit Cost Aggregation Sub-Architecture
• Disp R-CNN: Stereo 3D Object Detection via Shape Prior Guided Instance Disparity Estimation
• Learning monocular depth estimation infusing traditional stereo knowledge
• Self-Supervised Learning of Depth and Ego-motion with Differentiable Bundle Adjustment
• SENSE: a Shared Encoder Network for Scene-flow Estimation
• Deep Rigid Instance Scene Flow
• Group-wise Correlation Stereo Network
• Learning Independent Object Motion from Unlabelled Stereoscopic Videos
• Improving Deep Stereo Network Generalization with Geometric Priors
• Towards Practical Implementations of Person Re-Identification from Full Video Frames
• Learning Inverse Depth Regression for Multi-View Stereo with Correlation Cost Volume
• Learning to Detect 3D Reflection Symmetry for Single-View Reconstruction
• Hierarchical Deep Stereo Matching on High-resolution Images
• Flow2Stereo: Effective Self-Supervised Learning of Optical Flow and Stereo Matching
• Deep-Learning Assisted High-Resolution Binocular Stereo Depth Reconstruction
• Joint Learning of Semantic Alignment and Object Landmark Detection
• MTStereo 2.0: improved accuracy of stereo depth estimation withMax-trees
• StereoGAN: Bridging Synthetic-to-Real Domain Gap by Joint Optimization of Domain Translation and Stereo Matching
• Toward 3D Object Reconstruction from Stereo Images
• Deep Learning Stereo Vision at the edge
• Noise-Sampling Cross Entropy Loss: Improving Disparity Regression Via Cost Volume Aware Regularizer
• Multi-scale Cross-form Pyramid Network for Stereo Matching
• Learning Stereo Matchability in Disparity Regression Networks
• Increased-Range Unsupervised Monocular Depth Estimation
• A-TVSNet: Aggregated Two-View Stereo Network for Multi-View Stereo Depth Estimation
• Video Depth Estimation by Fusing Flow-to-Depth Proposals
• Learning Parallax Attention for Stereo Image Super-Resolution
• Depth by Poking: Learning to Estimate Depth from Self-Supervised Grasping