Acquiring a Dynamic Light Field through a Single-Shot Coded Image
arXiv:2204.12089 · doi:10.1109/CVPR52688.2022.01921
Abstract
We propose a method for compressively acquiring a dynamic light field (a 5-D volume) through a single-shot coded image (a 2-D measurement). We designed an imaging model that synchronously applies aperture coding and pixel-wise exposure coding within a single exposure time. This coding scheme enables us to effectively embed the original information into a single observed image. The observed image is then fed to a convolutional neural network (CNN) for light-field reconstruction, which is jointly trained with the camera-side coding patterns. We also developed a hardware prototype to capture a real 3-D scene moving over time. We succeeded in acquiring a dynamic light field with 5x5 viewpoints over 4 temporal sub-frames (100 views in total) from a single observed image. Repeating capture and reconstruction processes over time, we can acquire a dynamic light field at 4x the frame rate of the camera. To our knowledge, our method is the first to achieve a finer temporal resolution than the camera itself in compressive light-field acquisition. Our software is available from our project webpage
References in corpus (5)
- Stereo Magnification: Learning View Synthesis using Multiplane Images
- Light Field Video Capture Using a Learning-Based Hybrid Imaging System
- Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines
- Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Image
- Learning a Probabilistic Strategy for Computational Imaging Sensor Selection