NeRF++: Analyzing and Improving Neural Radiance Fields

Abstract

Neural Radiance Fields (NeRF) achieve impressive view synthesis results for a variety of capture settings, including 360 capture of bounded scenes and forward-facing capture of bounded and unbounded scenes. NeRF fits multi-layer perceptrons (MLPs) representing view-invariant opacity and view-dependent color volumes to a set of training images, and samples novel views based on volume rendering techniques. In this technical report, we first remark on radiance fields and their potential ambiguities, namely the shape-radiance ambiguity, and analyze NeRF's success in avoiding such ambiguities. Second, we address a parametrization issue involved in applying NeRF to 360 captures of objects within large-scale, unbounded 3D scenes. Our method improves view synthesis fidelity in this challenging scenario. Code is available at https://github.com/Kai-46/nerfplusplus.

Code is available at https://github.com/Kai-46/nerfplusplus; fix a minor formatting issue in Fig. 4

References in corpus (3)

• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• Neural Sparse Voxel Fields

Cited by in corpus (67)

• Fast Dynamic Radiance Fields with Time-Aware Neural Voxels
• DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks
• NeRF--: Neural Radiance Fields Without Known Camera Parameters
• StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
• Synthetic Datasets for Autonomous Driving: A Survey
• H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motion
• CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis
• Self-Supervised Coordinate Projection Network for Sparse-View Computed Tomography
• Style and Pose Control for Image Synthesis of Humans from a Single Monocular View
• Portrait Neural Radiance Fields from a Single Image
• UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution
• Fast Learning Radiance Fields by Shooting Much Fewer Rays
• Neural Volume Rendering: NeRF And Beyond
• DoF-NeRF: Depth-of-Field Meets Neural Radiance Fields
• SimpleNeRF: Regularizing Sparse Input Neural Radiance Fields with Simpler Solutions
• Neural Trajectory Fields for Dynamic Novel View Synthesis
• SqueezeNeRF: Further factorized FastNeRF for memory-efficient inference
• CLONeR: Camera-Lidar Fusion for Occupancy Grid-aided Neural Representations
• ADOP: Approximate Differentiable One-Pixel Point Rendering
• Remote Sensing Novel View Synthesis with Implicit Multiplane Representations
• A Survey on Deep Generative 3D-aware Image Synthesis
• Towards Open World NeRF-Based SLAM
• NeuLF: Efficient Novel View Synthesis with Neural 4D Light Field
• GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields
• Don't Splat your Gaussians: Volumetric Ray-Traced Primitives for Modeling and Rendering Scattering and Emissive Media
• FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold
• CoIL: Coordinate-based Internal Learning for Imaging Inverse Problems
• Implicit Ray-Transformers for Multi-view Remote Sensing Image Segmentation
• Zero-Shot Text-Guided Object Generation with Dream Fields
• Space-time Neural Irradiance Fields for Free-Viewpoint Video
• NeRF in detail: Learning to sample for view synthesis
• Crowd-Sourced NeRF: Collecting Data from Production Vehicles for 3D Street View Reconstruction
• NeuSample: Neural Sample Field for Efficient View Synthesis
• In-Place Scene Labelling and Understanding with Implicit Scene Representation
• Single-image Full-body Human Relighting
• Neural Knitworks: Patched Neural Implicit Representation Networks
• NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo
• Volume Rendering of Neural Implicit Surfaces
• NVS-MonoDepth: Improving Monocular Depth Prediction with Novel View Synthesis
• Geometric implicit neural representations for signed distance functions
• Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions
• Polynomial Neural Fields for Subband Decomposition and Manipulation
• Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering
• Direct-PoseNet: Absolute Pose Regression with Photometric Consistency
• Neural Fields in Visual Computing and Beyond
• EasyVolcap: Accelerating Neural Volumetric Video Research
• Reconstructing Continuous Light Field From Single Coded Image
• MM-NeRF: Multimodal-Guided 3D Multi-Style Transfer of Neural Radiance Field
• Recursive-NeRF: An Efficient and Dynamically Growing NeRF
• Deblur-NeRF: Neural Radiance Fields from Blurry Images
• Template NeRF: Towards Modeling Dense Shape Correspondences from Category-Specific Object Images
• BARF: Bundle-Adjusting Neural Radiance Fields
• Semantic Is Enough: Only Semantic Information For NeRF Reconstruction
• DoubleField: Bridging the Neural Surface and Radiance Fields for High-fidelity Human Reconstruction and Rendering
• LiveView: Dynamic Target-Centered MPI for View Synthesis
• Layout-Guided Novel View Synthesis from a Single Indoor Panorama
• MirrorNeRF: One-shot Neural Portrait Radiance Field from Multi-mirror Catadioptric Imaging
• Learning to Stylize Novel Views
• Convolutional Neural Opacity Radiance Fields
• STaR: Self-supervised Tracking and Reconstruction of Rigid Objects in Motion with Neural Rendering
• Self-Supervised Object Detection via Generative Image Synthesis
• Self-Supervised Visibility Learning for Novel View Synthesis
• PixelTransformer: Sample Conditioned Signal Generation
• MOSE: Monocular Semantic Reconstruction Using NeRF-Lifted Noisy Priors
• VaxNeRF: Revisiting the Classic for Voxel-Accelerated Neural Radiance Field
• Appearance Editing with Free-viewpoint Neural Rendering
• Point-Based Neural Rendering with Per-View Optimization