Fast Dynamic Radiance Fields with Time-Aware Neural Voxels
arXiv:2205.15285 · doi:10.1145/3550469.3555383
Abstract
Neural radiance fields (NeRF) have shown great success in modeling 3D scenes and synthesizing novel-view images. However, most previous NeRF methods take much time to optimize one single scene. Explicit data structures, e.g. voxel features, show great potential to accelerate the training process. However, voxel features face two big challenges to be applied to dynamic scenes, i.e. modeling temporal information and capturing different scales of point motions. We propose a radiance field framework by representing scenes with time-aware voxel features, named as TiNeuVox. A tiny coordinate deformation network is introduced to model coarse motion trajectories and temporal information is further enhanced in the radiance network. A multi-distance interpolation method is proposed and applied on voxel features to model both small and large motions. Our framework significantly accelerates the optimization of dynamic radiance fields while maintaining high rendering quality. Empirical evaluation is performed on both synthetic and real scenes. Our TiNeuVox completes training with only 8 minutes and 8-MB storage cost while showing similar or even better rendering performance than previous dynamic NeRF methods.
SIGGRAPH Asia 2022. Project page: https://jaminfong.cn/tineuvox
References in corpus (9)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
- NeRF++: Analyzing and Improving Neural Radiance Fields
- DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks
- NeRF--: Neural Radiance Fields Without Known Camera Parameters
- HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields
- H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motion
- Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction
- NeuSample: Neural Sample Field for Efficient View Synthesis
Cited by in corpus (20)
- HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion
- LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatar
- Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos
- Representing Long Volumetric Video with Temporal Gaussian Hierarchy
- Neural radiance fields in the industrial and robotics domain: applications, research opportunities and use cases
- GaussianPrediction: Dynamic 3D Gaussian Prediction for Motion Extrapolation and Free View Synthesis
- CTNeRF: Cross-Time Transformer for Dynamic Neural Radiance Field from Monocular Video
- Advances in Artificial Intelligence: A Review for the Creative Industries
- DynaMoN: Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields
- Dynamic NeRFs for Soccer Scenes
- Dynamic Appearance Particle Neural Radiance Field
- Factorized Motion Fields for Fast Sparse Input Dynamic View Synthesis
- Efficient 3D Reconstruction, Streaming and Visualization of Static and Dynamic Scene Parts for Multi-client Live-telepresence in Large-scale Environments
- FPO++: Efficient Encoding and Rendering of Dynamic Neural Radiance Fields by Analyzing and Enhancing Fourier PlenOctrees
- MoBluRF: Motion Deblurring Neural Radiance Fields for Blurry Monocular Video
- A Fast Volumetric Capture and Reconstruction Pipeline for Dynamic Point Clouds and Gaussian Splats
- TivNe-SLAM: Dynamic Mapping and Tracking via Time-Varying Neural Radiance Fields
- MultiEgo: A Multi-View Egocentric Video Dataset for 4D Scene Reconstruction
- High-quality Animatable Eyelid Shapes from Lightweight Captures
- Learning High-Fidelity Robot Self-Model with Articulated 3D Gaussian Splatting