Neural 3D Reconstruction in the Wild

Abstract

We are witnessing an explosion of neural implicit representations in computer vision and graphics. Their applicability has recently expanded beyond tasks such as shape generation and image-based rendering to the fundamental problem of image-based 3D reconstruction. However, existing methods typically assume constrained 3D environments with constant illumination captured by a small set of roughly uniformly distributed cameras. We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections in the presence of varying illumination. To achieve this, we propose a hybrid voxel- and surface-guided sampling technique that allows for more efficient ray sampling around surfaces and leads to significant improvements in reconstruction quality. Further, we present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes. We perform extensive experiments, demonstrating that our approach surpasses both classical and neural reconstruction methods on a wide variety of metrics.

Accepted to SIGGRAPH 2022 (Conference Proceedings). Project page: https://zju3dv.github.io/neuralrecon-w/

References in corpus (1)

• NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination

Cited by in corpus (5)

• IE-NeRF: Inpainting Enhanced Neural Radiance Fields in the Wild
• Methods and strategies for improving the novel view synthesis quality of neural radiation field
• MS-NeRF: Multi-Space Neural Radiance Fields
• TerrainMesh: Metric-Semantic Terrain Reconstruction from Aerial Images Using Joint 2D-3D Learning
• A Probability-guided Sampler for Neural Implicit Surface Rendering