Real-time Neural Radiance Caching for Path Tracing

Abstract

We present a real-time neural radiance caching method for path-traced global illumination. Our system is designed to handle fully dynamic scenes, and makes no assumptions about the lighting, geometry, and materials. The data-driven nature of our approach sidesteps many difficulties of caching algorithms, such as locating, interpolating, and updating cache points. Since pretraining neural networks to handle novel, dynamic scenes is a formidable generalization challenge, we do away with pretraining and instead achieve generalization via adaptation, i.e. we opt for training the radiance cache while rendering. We employ self-training to provide low-noise training targets and simulate infinite-bounce transport by merely iterating few-bounce training updates. The updates and cache queries incur a mild overhead -- about 2.6ms on full HD resolution -- thanks to a streaming implementation of the neural network that fully exploits modern hardware. We demonstrate significant noise reduction at the cost of little induced bias, and report state-of-the-art, real-time performance on a number of challenging scenarios.

arXiv commentTo appear at SIGGRAPH 2021. 16 pages, 16 figures

References in corpus (2)

• Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks
• Second-Order Occlusion-Aware Volumetric Radiance Caching

Cited by in corpus (17)

• Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
• 3D Gaussian as a New Era: A Survey
• Efficient physics-informed neural networks using hash encoding
• Neural Radiosity
• Fast Neural Representations for Direct Volume Rendering
• Random-Access Neural Compression of Material Textures
• Online Neural Path Guiding with Normalized Anisotropic Spherical Gaussians
• Neural Parametric Mixtures for Path Guiding
• Learning to Learn and Sample BRDFs
• N-Dimensional Gaussians for Fitting of High Dimensional Functions
• Real-time Neural Woven Fabric Rendering
• RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illumination
• Efficient Graphics Representation with Differentiable Indirection
• Efficient Neural Representation of Volumetric Data using Coordinate-Based Networks
• MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields
• Residual path integrals for re-rendering
• Physically-based Path Tracer using WebGPU and OpenPBR