DoF-NeRF: Depth-of-Field Meets Neural Radiance Fields
arXiv:2208.00945 · doi:10.1145/3503161.3548088
Abstract
Neural Radiance Field (NeRF) and its variants have exhibited great success on representing 3D scenes and synthesizing photo-realistic novel views. However, they are generally based on the pinhole camera model and assume all-in-focus inputs. This limits their applicability as images captured from the real world often have finite depth-of-field (DoF). To mitigate this issue, we introduce DoF-NeRF, a novel neural rendering approach that can deal with shallow DoF inputs and can simulate DoF effect. In particular, it extends NeRF to simulate the aperture of lens following the principles of geometric optics. Such a physical guarantee allows DoF-NeRF to operate views with different focus configurations. Benefiting from explicit aperture modeling, DoF-NeRF also enables direct manipulation of DoF effect by adjusting virtual aperture and focus parameters. It is plug-and-play and can be inserted into NeRF-based frameworks. Experiments on synthetic and real-world datasets show that, DoF-NeRF not only performs comparably with NeRF in the all-in-focus setting, but also can synthesize all-in-focus novel views conditioned on shallow DoF inputs. An interesting application of DoF-NeRF to DoF rendering is also demonstrated. The source code will be made available at https://github.com/zijinwuzijin/DoF-NeRF.
arXiv commentAccepted by ACMMM 2022
References in corpus (12)
- Neural Volumes: Learning Dynamic Renderable Volumes from Images
- NeRF++: Analyzing and Improving Neural Radiance Fields
- GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
- Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Images
- Synthetic Depth-of-Field with a Single-Camera Mobile Phone
- Stereo Magnification: Learning View Synthesis using Multiplane Images
- Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines
- Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction
- BGGAN: Bokeh-Glass Generative Adversarial Network for Rendering Realistic Bokeh
- NeRFocus: Neural Radiance Field for 3D Synthetic Defocus
- HDR-NeRF: High Dynamic Range Neural Radiance Fields
- Synthetic Defocus and Look-Ahead Autofocus for Casual Videography