Light Field Image Super-Resolution Using Deformable Convolution

Abstract

Light field (LF) cameras can record scenes from multiple perspectives, and thus introduce beneficial angular information for image super-resolution (SR). However, it is challenging to incorporate angular information due to disparities among LF images. In this paper, we propose a deformable convolution network (i.e., LF-DFnet) to handle the disparity problem for LF image SR. Specifically, we design an angular deformable alignment module (ADAM) for feature-level alignment. Based on ADAM, we further propose a collect-and-distribute approach to perform bidirectional alignment between the center-view feature and each side-view feature. Using our approach, angular information can be well incorporated and encoded into features of each view, which benefits the SR reconstruction of all LF images. Moreover, we develop a baseline-adjustable LF dataset to evaluate SR performance under different disparity variations. Experiments on both public and our self-developed datasets have demonstrated the superiority of our method. Our LF-DFnet can generate high-resolution images with more faithful details and achieve state-of-the-art reconstruction accuracy. Besides, our LF-DFnet is more robust to disparity variations, which has not been well addressed in literature.

Accepted by IEEE Transactions on Image Processing

References in corpus (10)

• Deep Learning for Single Image Super-Resolution: A Brief Review
• Lightweight Image Super-Resolution with Information Multi-distillation Network
• ESRGAN+ : Further Improving Enhanced Super-Resolution Generative Adversarial Network
• Deformable 3D Convolution for Video Super-Resolution
• Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications
• Selective Light Field Refocusing for Camera Arrays Using Bokeh Rendering and Superresolution
• Light Field Super-Resolution Via Graph-Based Regularization
• No-Reference Light Field Image Quality Assessment Based on Spatial-Angular Measurement
• Spatial and Angular Resolution Enhancement of Light Fields Using Convolutional Neural Networks
• Deformable Non-local Network for Video Super-Resolution

Cited by in corpus (4)

• Disentangling Light Fields for Super-Resolution and Disparity Estimation
• Light Field Image Super-Resolution with Transformers
• Dense Dual-Attention Network for Light Field Image Super-Resolution
• Probabilistic-based Feature Embedding of 4-D Light Fields for Compressive Imaging and Denoising