Light Field Super-Resolution Via Graph-Based Regularization

Abstract

Light field cameras capture the 3D information in a scene with a single exposure. This special feature makes light field cameras very appealing for a variety of applications: from post-capture refocus, to depth estimation and image-based rendering. However, light field cameras suffer by design from strong limitations in their spatial resolution, which should therefore be augmented by computational methods. On the one hand, off-the-shelf single-frame and multi-frame super-resolution algorithms are not ideal for light field data, as they do not consider its particular structure. On the other hand, the few super-resolution algorithms explicitly tailored for light field data exhibit significant limitations, such as the need to estimate an explicit disparity map at each view. In this work we propose a new light field super-resolution algorithm meant to address these limitations. We adopt a multi-frame alike super-resolution approach, where the complementary information in the different light field views is used to augment the spatial resolution of the whole light field. We show that coupling the multi-frame approach with a graph regularizer, that enforces the light field structure via nonlocal self similarities, permits to avoid the costly and challenging disparity estimation step for all the views. Extensive experiments show that the new algorithm compares favorably to the other state-of-the-art methods for light field super-resolution, both in terms of PSNR and visual quality.

This new version includes more material. In particular, we added: a new section on the computational complexity of the proposed algorithm, experimental comparisons with a CNN-based super-resolution algorithm, and new experiments on a third dataset

References in corpus (1)

• Light Field Super-Resolution Via Graph-Based Regularization

Cited by in corpus (9)

• Disentangling Light Fields for Super-Resolution and Disparity Estimation
• Light Field Image Super-Resolution Using Deformable Convolution
• High-dimensional Dense Residual Convolutional Neural Network for Light Field Reconstruction
• Light Field Super-Resolution Via Graph-Based Regularization
• Dense Dual-Attention Network for Light Field Image Super-Resolution
• Beyond Subspace Isolation: Many-to-Many Transformer for Light Field Image Super-resolution
• 3DVSR: 3D EPI Volume-based Approach for Angular and Spatial Light field Image Super-resolution
• Light Field Spatial Super-resolution via Deep Combinatorial Geometry Embedding and Structural Consistency Regularization
• Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses