Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching

Abstract

The deep multi-view stereo (MVS) and stereo matching approaches generally construct 3D cost volumes to regularize and regress the output depth or disparity. These methods are limited when high-resolution outputs are needed since the memory and time costs grow cubically as the volume resolution increases. In this paper, we propose a both memory and time efficient cost volume formulation that is complementary to existing multi-view stereo and stereo matching approaches based on 3D cost volumes. First, the proposed cost volume is built upon a standard feature pyramid encoding geometry and context at gradually finer scales. Then, we can narrow the depth (or disparity) range of each stage by the depth (or disparity) map from the previous stage. With gradually higher cost volume resolution and adaptive adjustment of depth (or disparity) intervals, the output is recovered in a coarser to fine manner. We apply the cascade cost volume to the representative MVS-Net, and obtain a 23.1% improvement on DTU benchmark (1st place), with 50.6% and 74.2% reduction in GPU memory and run-time. It is also the state-of-the-art learning-based method on Tanks and Temples benchmark. The statistics of accuracy, run-time and GPU memory on other representative stereo CNNs also validate the effectiveness of our proposed method.

Accepted by CVPR2020 Oral

References in corpus (3)

• Computing the Stereo Matching Cost with a Convolutional Neural Network
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• Multi-View Stereo with Asymmetric Checkerboard Propagation and Multi-Hypothesis Joint View Selection

Cited by in corpus (12)

• SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi-view Stereopsis
• Deep Learning for Multi-View Stereo via Plane Sweep: A Survey
• Deep Multi-View Stereo gone wild
• AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching
• CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching
• Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking
• M^3VSNet: Unsupervised Multi-metric Multi-view Stereo Network
• Bilateral Grid Learning for Stereo Matching Networks
• NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video
• Efficient Neural Radiance Fields for Interactive Free-viewpoint Video
• MeshMVS: Multi-View Stereo Guided Mesh Reconstruction
• A Decomposition Model for Stereo Matching