Untrained networks for compressive lensless photography

Abstract

Compressive lensless imagers enable novel applications in an extremely compact device, requiring only a phase or amplitude mask placed close to the sensor. They have been demonstrated for 2D and 3D microscopy, single-shot video, and single-shot hyperspectral imaging; in each of these cases, a compressive-sensing-based inverse problem is solved in order to recover a 3D data-cube from a 2D measurement. Typically, this is accomplished using convex optimization and hand-picked priors. Alternatively, deep learning-based reconstruction methods offer the promise of better priors, but require many thousands of ground truth training pairs, which can be difficult or impossible to acquire. In this work, we propose the use of untrained networks for compressive image recovery. Our approach does not require any labeled training data, but instead uses the measurement itself to update the network weights. We demonstrate our untrained approach on lensless compressive 2D imaging as well as single-shot high-speed video recovery using the camera's rolling shutter, and single-shot hyperspectral imaging. We provide simulation and experimental verification, showing that our method results in improved image quality over existing methods.

arXiv comment17 pages, 8 figures

References in corpus (13)

• Deep Convolutional Neural Network for Inverse Problems in Imaging
• A Review of Convolutional Neural Networks for Inverse Problems in Imaging
• Coded aperture compressive temporal imaging
• Imaging With Nature: Compressive Imaging Using a Multiply Scattering Medium
• Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array
• Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks
• Learned reconstructions for practical mask-based lensless imaging
• Compressed Sensing with Deep Image Prior and Learned Regularization
• Miniscope3D: optimized single-shot miniature 3D fluorescence microscopy
• Diffraction tomography with a deep image prior
• Plug-and-Play Methods Provably Converge with Properly Trained Denoisers
• FlatNet: Towards Photorealistic Scene Reconstruction from Lensless Measurements
• Fourier DiffuserScope: Single-shot 3D Fourier light field microscopy with a diffuser

Cited by in corpus (6)

• Untrained, physics-informed neural networks for structured illumination microscopy
• Res-U2Net: Untrained Deep Learning for Phase Retrieval and Image Reconstruction
• Extended Depth-of-Field Lensless Imaging using an Optimized Radial Mask
• Designing lensless imaging systems to maximize information capture
• Coded Illumination for Improved Lensless Imaging
• Coded Illumination for 3D Lensless Imaging