Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance
arXiv:2003.09852
Abstract
In this work we address the challenging problem of multiview 3D surface reconstruction. We introduce a neural network architecture that simultaneously learns the unknown geometry, camera parameters, and a neural renderer that approximates the light reflected from the surface towards the camera. The geometry is represented as a zero level-set of a neural network, while the neural renderer, derived from the rendering equation, is capable of (implicitly) modeling a wide set of lighting conditions and materials. We trained our network on real world 2D images of objects with different material properties, lighting conditions, and noisy camera initializations from the DTU MVS dataset. We found our model to produce state of the art 3D surface reconstructions with high fidelity, resolution and detail.
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
- Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer
- Differentiable Surface Splatting for Point-based Geometry Processing
- PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
- The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies
- Learning to Infer Implicit Surfaces without 3D Supervision
- Soft Rasterizer: Differentiable Rendering for Unsupervised Single-View Mesh Reconstruction
- Controlling Neural Level Sets
- Implicit Geometric Regularization for Learning Shapes
- DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing
- SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization
Cited by in corpus (9)
- Differentiable Rendering: A Survey
- 3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting
- Neural Volume Rendering: NeRF And Beyond
- Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with Whitted-Style Ray Tracing
- Benchmarking Neural Radiance Fields for Autonomous Robots: An Overview
- Physics-Based Inverse Rendering using Combined Implicit and Explicit Geometries
- Neural Parameterization for Dynamic Human Head Editing
- Normal-guided Detail-Preserving Neural Implicit Function for High-Fidelity 3D Surface Reconstruction
- A Probability-guided Sampler for Neural Implicit Surface Rendering