SegStereo: Exploiting Semantic Information for Disparity Estimation
arXiv:1807.11699
Abstract
Disparity estimation for binocular stereo images finds a wide range of applications. Traditional algorithms may fail on featureless regions, which could be handled by high-level clues such as semantic segments. In this paper, we suggest that appropriate incorporation of semantic cues can greatly rectify prediction in commonly-used disparity estimation frameworks. Our method conducts semantic feature embedding and regularizes semantic cues as the loss term to improve learning disparity. Our unified model SegStereo employs semantic features from segmentation and introduces semantic softmax loss, which helps improve the prediction accuracy of disparity maps. The semantic cues work well in both unsupervised and supervised manners. SegStereo achieves state-of-the-art results on KITTI Stereo benchmark and produces decent prediction on both CityScapes and FlyingThings3D datasets.
arXiv commentAccepted to ECCV 2018
References in corpus (10)
- Deep Residual Learning for Image Recognition
- Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Pyramid Scene Parsing Network
- Semi-Supervised Deep Learning for Monocular Depth Map Prediction
- DeepStereo: Learning to Predict New Views from the World's Imagery
- Pyramid Stereo Matching Network
- Learning for Disparity Estimation through Feature Constancy
- Deep Stereo Matching with Explicit Cost Aggregation Sub-Architecture
- DeepMatching: Hierarchical Deformable Dense Matching