Pyramid Stereo Matching Network

Abstract

Recent work has shown that depth estimation from a stereo pair of images can be formulated as a supervised learning task to be resolved with convolutional neural networks (CNNs). However, current architectures rely on patch-based Siamese networks, lacking the means to exploit context information for finding correspondence in illposed regions. To tackle this problem, we propose PSMNet, a pyramid stereo matching network consisting of two main modules: spatial pyramid pooling and 3D CNN. The spatial pyramid pooling module takes advantage of the capacity of global context information by aggregating context in different scales and locations to form a cost volume. The 3D CNN learns to regularize cost volume using stacked multiple hourglass networks in conjunction with intermediate supervision. The proposed approach was evaluated on several benchmark datasets. Our method ranked first in the KITTI 2012 and 2015 leaderboards before March 18, 2018. The codes of PSMNet are available at: https://github.com/JiaRenChang/PSMNet.

arXiv commentCVPR 2018

References in corpus (15)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Deep Residual Learning for Image Recognition
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Multi-Scale Context Aggregation by Dilated Convolutions
• ParseNet: Looking Wider to See Better
• Fully Convolutional Networks for Semantic Segmentation
• Stacked Hourglass Networks for Human Pose Estimation
• Pyramid Scene Parsing Network
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
• Stacked Deconvolutional Network for Semantic Segmentation
• Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching
• Label Refinement Network for Coarse-to-Fine Semantic Segmentation
• Learning for Disparity Estimation through Feature Constancy

Cited by in corpus (27)

• Hierarchical Neural Architecture Search for Deep Stereo Matching
• SERV-CT: A disparity dataset from CT for validation of endoscopic 3D reconstruction
• Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
• Learning Monocular Depth by Distilling Cross-domain Stereo Networks
• Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference
• Joint Unsupervised Learning of Optical Flow and Depth by Watching Stereo Videos
• CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching
• Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimation
• Boosting Real-Time Driving Scene Parsing with Shared Semantics
• Joint Pruning & Quantization for Extremely Sparse Neural Networks
• ResFPN: Residual Skip Connections in Multi-Resolution Feature Pyramid Networks for Accurate Dense Pixel Matching
• YOLOStereo3D: A Step Back to 2D for Efficient Stereo 3D Detection
• SegStereo: Exploiting Semantic Information for Disparity Estimation
• Real-Time Dense Stereo Embedded in A UAV for Road Inspection
• OmniMVS: End-to-End Learning for Omnidirectional Stereo Matching
• Learning Dense Stereo Matching for Digital Surface Models from Satellite Imagery
• Real-time Scalable Dense Surfel Mapping
• Normal Assisted Stereo Depth Estimation
• CodedStereo: Learned Phase Masks for Large Depth-of-field Stereo
• MSDC-Net: Multi-Scale Dense and Contextual Networks for Automated Disparity Map for Stereo Matching
• Multi-scale Cross-form Pyramid Network for Stereo Matching
• A Decomposition Model for Stereo Matching
• RRNet: Repetition-Reduction Network for Energy Efficient Decoder of Depth Estimation
• CNN based dense underwater 3D scene reconstruction by transfer learning using bubble database
• Stereo Waterdrop Removal with Row-wise Dilated Attention
• Multi-scale Iterative Residuals for Fast and Scalable Stereo Matching
• Shift Convolution Network for Stereo Matching