NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
arXiv:2003.08934
Abstract
We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene using a fully-connected (non-convolutional) deep network, whose input is a single continuous 5D coordinate (spatial location and viewing direction ) and whose output is the volume density and view-dependent emitted radiance at that spatial location. We synthesize views by querying 5D coordinates along camera rays and use classic volume rendering techniques to project the output colors and densities into an image. Because volume rendering is naturally differentiable, the only input required to optimize our representation is a set of images with known camera poses. We describe how to effectively optimize neural radiance fields to render photorealistic novel views of scenes with complicated geometry and appearance, and demonstrate results that outperform prior work on neural rendering and view synthesis. View synthesis results are best viewed as videos, so we urge readers to view our supplementary video for convincing comparisons.
ECCV 2020 (oral). Project page with videos and code: http://tancik.com/nerf
Cited by in corpus (49)
- Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
- Implicit Neural Representations with Periodic Activation Functions
- DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks
- StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
- Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance
- NeRF2: Neural Radio-Frequency Radiance Fields
- Neural Reflectance Fields for Appearance Acquisition
- H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motion
- CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory
- Learning Deformable Tetrahedral Meshes for 3D Reconstruction
- Learning Continuous Image Representation with Local Implicit Image Function
- NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation framework
- Fisher Information Field: an Efficient and Differentiable Map for Perception-aware Planning
- COIN: COmpression with Implicit Neural representations
- PVA: Pixel-aligned Volumetric Avatars
- Vid2Actor: Free-viewpoint Animatable Person Synthesis from Video in the Wild
- Equivariant Neural Rendering
- SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images
- ShapeFlow: Learnable Deformations Among 3D Shapes
- Learning Pose-invariant 3D Object Reconstruction from Single-view Images
- Neural Sparse Voxel Fields
- CKConv: Continuous Kernel Convolution For Sequential Data
- CreepyCoCreator? Investigating AI Representation Modes for 3D Object Co-Creation in Virtual Reality
- Deep Point Cloud Reconstruction
- Reproducing Activation Function for Deep Learning
- Predicting Infectiousness for Proactive Contact Tracing
- Revisiting Latent-Space Interpolation via a Quantitative Evaluation Framework
- NeRD: Neural Representation of Distribution for Medical Image Segmentation
- Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures
- When Gaussian Meets Surfel: Ultra-fast High-fidelity Radiance Field Rendering
- CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translation
- Study of the asteroid Bennu using geodesyANNs and Osiris-Rex data
- Auto-Tuned Sim-to-Real Transfer
- IntraSeismic: a coordinate-based learning approach to seismic inversion
- S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling
- CodeNeRF: Disentangled Neural Radiance Fields for Object Categories
- Context-Aware Indoor Point Cloud Object Generation through User Instructions
- Neural Rays for Occlusion-aware Image-based Rendering
- LiveView: Dynamic Target-Centered MPI for View Synthesis
- ImpliCity: City Modeling from Satellite Images with Deep Implicit Occupancy Fields
- Real-time Light Estimation and Neural Soft Shadows for AR Indoor Scenarios
- Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting
- Dynamic Neural Garments
- Length Scale Control in Topology Optimization using Fourier Enhanced Neural Networks
- Manipulating Vehicle 3D Shapes through Latent Space Editing
- Neural Camera Simulators
- Towards Generalising Neural Implicit Representations
- CNN with large memory layers
- Towards Lightweight Controllable Audio Synthesis with Conditional Implicit Neural Representations