Spatial-Angular Attention Network for Light Field Reconstruction

Abstract

Typical learning-based light field reconstruction methods demand in constructing a large receptive field by deepening the network to capture correspondences between input views. In this paper, we propose a spatial-angular attention network to perceive correspondences in the light field non-locally, and reconstruction high angular resolution light field in an end-to-end manner. Motivated by the non-local attention mechanism, a spatial-angular attention module specifically for the high-dimensional light field data is introduced to compute the responses from all the positions in the epipolar plane for each pixel in the light field, and generate an attention map that captures correspondences along the angular dimension. We then propose a multi-scale reconstruction structure to efficiently implement the non-local attention in the low spatial scale, while also preserving the high frequency components in the high spatial scales. Extensive experiments demonstrate the superior performance of the proposed spatial-angular attention network for reconstructing sparsely-sampled light fields with non-Lambertian effects.

arXiv comment15 pages, 13 figures and 5 tables, Accepted by IEEE Transactions on Image Processing (IEEE TIP)

References in corpus (14)

• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Deep Convolutional Neural Network for Inverse Problems in Imaging
• Self-Attention Generative Adversarial Networks
• Learning-Based View Synthesis for Light Field Cameras
• HDR image reconstruction from a single exposure using deep CNNs
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications
• A System for Acquiring, Processing, and Rendering Panoramic Light Field Stills for Virtual Reality
• Do Convnets Learn Correspondence?
• High-dimensional Dense Residual Convolutional Neural Network for Light Field Reconstruction
• Towards a quality metric for dense light fields
• Revisiting Light Field Rendering with Deep Anti-Aliasing Neural Network
• FcaNet: Frequency Channel Attention Networks
• Light Field View Synthesis via Aperture Disparity and Warping Confidence Map

Cited by in corpus (2)

• Disentangling Light Fields for Super-Resolution and Disparity Estimation
• Light Field Implicit Representation for Flexible Resolution Reconstruction