Point-Based Multi-View Stereo Network

Abstract

We introduce Point-MVSNet, a novel point-based deep framework for multi-view stereo (MVS). Distinct from existing cost volume approaches, our method directly processes the target scene as point clouds. More specifically, our method predicts the depth in a coarse-to-fine manner. We first generate a coarse depth map, convert it into a point cloud and refine the point cloud iteratively by estimating the residual between the depth of the current iteration and that of the ground truth. Our network leverages 3D geometry priors and 2D texture information jointly and effectively by fusing them into a feature-augmented point cloud, and processes the point cloud to estimate the 3D flow for each point. This point-based architecture allows higher accuracy, more computational efficiency and more flexibility than cost-volume-based counterparts. Experimental results show that our approach achieves a significant improvement in reconstruction quality compared with state-of-the-art methods on the DTU and the Tanks and Temples dataset. Our source code and trained models are available at https://github.com/callmeray/PointMVSNet .

Accepted as ICCV 2019 oral presentation

References in corpus (7)

• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Dynamic Graph CNN for Learning on Point Clouds
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• BA-Net: Dense Bundle Adjustment Network
• MVSNet: Depth Inference for Unstructured Multi-view Stereo
• DeepMVS: Learning Multi-view Stereopsis

Cited by in corpus (2)

• A Survey of Simultaneous Localization and Mapping with an Envision in 6G Wireless Networks
• DSGN: Deep Stereo Geometry Network for 3D Object Detection