DeepMVS: Learning Multi-view Stereopsis

Abstract

We present DeepMVS, a deep convolutional neural network (ConvNet) for multi-view stereo reconstruction. Taking an arbitrary number of posed images as input, we first produce a set of plane-sweep volumes and use the proposed DeepMVS network to predict high-quality disparity maps. The key contributions that enable these results are (1) supervised pretraining on a photorealistic synthetic dataset, (2) an effective method for aggregating information across a set of unordered images, and (3) integrating multi-layer feature activations from the pre-trained VGG-19 network. We validate the efficacy of DeepMVS using the ETH3D Benchmark. Our results show that DeepMVS compares favorably against state-of-the-art conventional MVS algorithms and other ConvNet based methods, particularly for near-textureless regions and thin structures.

arXiv commentCVPR 2018. Project page: https://phuang17.github.io/DeepMVS/ Code: https://github.com/phuang17/DeepMVS

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Fully Convolutional Networks for Semantic Segmentation
• DeMoN: Depth and Motion Network for Learning Monocular Stereo
• Rich feature hierarchies for accurate object detection and semantic segmentation
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• Learning to Compare Image Patches via Convolutional Neural Networks
• Playing for Data: Ground Truth from Computer Games
• DeepStereo: Learning to Predict New Views from the World's Imagery

Cited by in corpus (10)

• Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction
• Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation
• Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference
• Point-Based Multi-View Stereo Network
• DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency
• Normal Assisted Stereo Depth Estimation
• CodeMapping: Real-Time Dense Mapping for Sparse SLAM using Compact Scene Representations
• CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency
• Multi-view 3D Reconstruction with Transformer
• Associative3D: Volumetric Reconstruction from Sparse Views