SeamlessNeRF: Stitching Part NeRFs with Gradient Propagation

Abstract

Neural Radiance Fields (NeRFs) have emerged as promising digital mediums of 3D objects and scenes, sparking a surge in research to extend the editing capabilities in this domain. The task of seamless editing and merging of multiple NeRFs, resembling the ``Poisson blending'' in 2D image editing, remains a critical operation that is under-explored by existing work. To fill this gap, we propose SeamlessNeRF, a novel approach for seamless appearance blending of multiple NeRFs. In specific, we aim to optimize the appearance of a target radiance field in order to harmonize its merge with a source field. We propose a well-tailored optimization procedure for blending, which is constrained by 1) pinning the radiance color in the intersecting boundary area between the source and target fields and 2) maintaining the original gradient of the target. Extensive experiments validate that our approach can effectively propagate the source appearance from the boundary area to the entire target field through the gradients. To the best of our knowledge, SeamlessNeRF is the first work that introduces gradient-guided appearance editing to radiance fields, offering solutions for seamless stitching of 3D objects represented in NeRFs.

arXiv commentTo appear in SIGGRAPH Asia 2023. Project website is accessible at https://sites.google.com/view/seamlessnerf

References in corpus (11)

• Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
• Adding Conditional Control to Text-to-Image Diffusion Models
• Decomposing NeRF for Editing via Feature Field Distillation
• Editable Free-viewpoint Video Using a Layered Neural Representation
• SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections
• Compressible-composable NeRF via Rank-residual Decomposition
• Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction
• UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene
• Signal Processing for Implicit Neural Representations
• ARF: Artistic Radiance Fields
• NeRFEditor: Differentiable Style Decomposition for Full 3D Scene Editing