NeRF-NQA: No-Reference Quality Assessment for Scenes Generated by NeRF and Neural View Synthesis Methods

Abstract

Neural View Synthesis (NVS) has demonstrated efficacy in generating high-fidelity dense viewpoint videos using a image set with sparse views. However, existing quality assessment methods like PSNR, SSIM, and LPIPS are not tailored for the scenes with dense viewpoints synthesized by NVS and NeRF variants, thus, they often fall short in capturing the perceptual quality, including spatial and angular aspects of NVS-synthesized scenes. Furthermore, the lack of dense ground truth views makes the full reference quality assessment on NVS-synthesized scenes challenging. For instance, datasets such as LLFF provide only sparse images, insufficient for complete full-reference assessments. To address the issues above, we propose NeRF-NQA, the first no-reference quality assessment method for densely-observed scenes synthesized from the NVS and NeRF variants. NeRF-NQA employs a joint quality assessment strategy, integrating both viewwise and pointwise approaches, to evaluate the quality of NVS-generated scenes. The viewwise approach assesses the spatial quality of each individual synthesized view and the overall inter-views consistency, while the pointwise approach focuses on the angular qualities of scene surface points and their compound inter-point quality. Extensive evaluations are conducted to compare NeRF-NQA with 23 mainstream visual quality assessment methods (from fields of image, video, and light-field assessment). The results demonstrate NeRF-NQA outperforms the existing assessment methods significantly and it shows substantial superiority on assessing NVS-synthesized scenes without references. An implementation of this paper are available at https://github.com/VincentQQu/NeRF-NQA.

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
• Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index
• Image Quality Assessment: Unifying Structure and Texture Similarity
• A Haar Wavelet-Based Perceptual Similarity Index for Image Quality Assessment
• Mean Deviation Similarity Index: Efficient and Reliable Full-Reference Image Quality Evaluator
• Light Field Image Quality Assessment With Auxiliary Learning Based on Depthwise and Anglewise Separable Convolutions
• LFACon: Introducing Anglewise Attention to No-Reference Quality Assessment in Light Field Space

Cited by in corpus (3)

• Towards End-to-End Neuromorphic Event-based 3D Object Reconstruction Without Physical Priors
• LoDAvatar: Hierarchical Embedding and Selective Detail Enhancement for Adaptive Levels of Detail Gaussian Avatars
• Quality Assessment of 3D Gaussian Splatting: Distortions, Benchmarks, and Open Challenges