NeRF--: Neural Radiance Fields Without Known Camera Parameters
arXiv:2102.07064
Abstract
Considering the problem of novel view synthesis (NVS) from only a set of 2D images, we simplify the training process of Neural Radiance Field (NeRF) on forward-facing scenes by removing the requirement of known or pre-computed camera parameters, including both intrinsics and 6DoF poses. To this end, we propose NeRF, with three contributions: First, we show that the camera parameters can be jointly optimised as learnable parameters with NeRF training, through a photometric reconstruction; Second, to benchmark the camera parameter estimation and the quality of novel view renderings, we introduce a new dataset of path-traced synthetic scenes, termed as Blender Forward-Facing Dataset (BLEFF); Third, we conduct extensive analyses to understand the training behaviours under various camera motions, and show that in most scenarios, the joint optimisation pipeline can recover accurate camera parameters and achieve comparable novel view synthesis quality as those trained with COLMAP pre-computed camera parameters. Our code and data are available at https://nerfmm.active.vision.
Project page see https://nerfmm.active.vision. Add a break point analysis experiment and release a BLEFF dataset
Cited by in corpus (27)
- Fast Dynamic Radiance Fields with Time-Aware Neural Voxels
- CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis
- Artemis: Articulated Neural Pets with Appearance and Motion synthesis
- Long-term Dependency for 3D Reconstruction of Freehand Ultrasound Without External Tracker
- NeuLF: Efficient Novel View Synthesis with Neural 4D Light Field
- Towards Open World NeRF-Based SLAM
- A Survey on Deep Generative 3D-aware Image Synthesis
- NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images
- Benchmarking Neural Radiance Fields for Autonomous Robots: An Overview
- MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal calibration
- AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
- NeRF in detail: Learning to sample for view synthesis
- Crowd-Sourced NeRF: Collecting Data from Production Vehicles for 3D Street View Reconstruction
- SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decomposition
- SlimmeRF: Slimmable Radiance Fields
- Direct-PoseNet: Absolute Pose Regression with Photometric Consistency
- Efficient 3D Reconstruction, Streaming and Visualization of Static and Dynamic Scene Parts for Multi-client Live-telepresence in Large-scale Environments
- CAD-NeRF: Learning NeRFs from Uncalibrated Few-view Images by CAD Model Retrieval
- Neural Fields in Visual Computing and Beyond
- CodeNeRF: Disentangled Neural Radiance Fields for Object Categories
- Generative Object Insertion in Gaussian Splatting with a Multi-View Diffusion Model
- Deblur-NeRF: Neural Radiance Fields from Blurry Images
- ImplicitVol: Sensorless 3D Ultrasound Reconstruction with Deep Implicit Representation
- BARF: Bundle-Adjusting Neural Radiance Fields
- NoPe-NeRF++: Local-to-Global Optimization of NeRF with No Pose Prior
- Learning to Stylize Novel Views
- In-Hand 3D Object Scanning from an RGB Sequence