SlimmeRF: Slimmable Radiance Fields

Abstract

Neural Radiance Field (NeRF) and its variants have recently emerged as successful methods for novel view synthesis and 3D scene reconstruction. However, most current NeRF models either achieve high accuracy using large model sizes, or achieve high memory-efficiency by trading off accuracy. This limits the applicable scope of any single model, since high-accuracy models might not fit in low-memory devices, and memory-efficient models might not satisfy high-quality requirements. To this end, we present SlimmeRF, a model that allows for instant test-time trade-offs between model size and accuracy through slimming, thus making the model simultaneously suitable for scenarios with different computing budgets. We achieve this through a newly proposed algorithm named Tensorial Rank Incrementation (TRaIn) which increases the rank of the model's tensorial representation gradually during training. We also observe that our model allows for more effective trade-offs in sparse-view scenarios, at times even achieving higher accuracy after being slimmed. We credit this to the fact that erroneous information such as floaters tend to be stored in components corresponding to higher ranks. Our implementation is available at https://github.com/Shiran-Yuan/SlimmeRF.

3DV 2024 Oral, Project Page: https://shiran-yuan.github.io/SlimmeRF/, Code: https://github.com/Shiran-Yuan/SlimmeRF/

References in corpus (10)

• Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
• DreamFusion: Text-to-3D using 2D Diffusion
• GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
• NeRF--: Neural Radiance Fields Without Known Camera Parameters
• StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
• MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Driving
• Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors
• Compressing Explicit Voxel Grid Representations: fast NeRFs become also small
• Compressible-composable NeRF via Rank-residual Decomposition
• D-TensoRF: Tensorial Radiance Fields for Dynamic Scenes