Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media

Abstract

Imaging through scattering is an important, yet challenging problem. Tremendous progress has been made by exploiting the deterministic input-output "transmission matrix" for a fixed medium. However, this "one-to-one" mapping is highly susceptible to speckle decorrelations - small perturbations to the scattering medium lead to model errors and severe degradation of the imaging performance. Our goal here is to develop a new framework that is highly scalable to both medium perturbations and measurement requirement. To do so, we propose a statistical "one-to-all" deep learning technique that encapsulates a wide range of statistical variations for the model to be resilient to speckle decorrelations. Specifically, we develop a convolutional neural network (CNN) that is able to learn the statistical information contained in the speckle intensity patterns captured on a set of diffusers having the same macroscopic parameter. We then show for the first time, to the best of our knowledge, that the trained CNN is able to generalize and make high-quality object predictions through an entirely different set of diffusers of the same class. Our work paves the way to a highly scalable deep learning approach for imaging through scattering media.

References in corpus (13)

• Measuring the Transmission Matrix in Optics : An Approach to the Study and Control of Light Propagation in Disordered Media
• Non-invasive real-time imaging through scattering layers and around corners via speckle correlations
• Phase recovery and holographic image reconstruction using deep learning in neural networks
• Deep Learning Microscopy
• Light fields in complex media: mesoscopic scattering meets wave control
• Learning to see through multimode fibers
• Reference-less measurement of the transmission matrix of a highly scattering material using a DMD and phase retrieval techniques
• Characterization of the angular memory effect of scattered light in biological tissues
• Online learning of the transfer matrix of dynamic scattering media: wavefront shaping meets multidimensional time series
• Efficient and accurate inversion of multiple scattering with deep learning
• Focusing light through dynamical samples using fast closed-loop wavefront optimization
• Field of view advantage of conjugate adaptive optics in microscopy applications
• Efficient Inversion of Multiple-Scattering Model for Optical Diffraction Tomography

Cited by in corpus (56)

• Deep learning in nano-photonics: inverse design and beyond
• Design of Task-Specific Optical Systems Using Broadband Diffractive Neural Networks
• Reliable deep-learning-based phase imaging with uncertainty quantification
• Roadmap on Wavefront Shaping and deep imaging in complex media
• Computational Imaging Without a Computer: Seeing Through Random Diffusers at the Speed of Light
• Learned reconstructions for practical mask-based lensless imaging
• Learning to decompose the modes in few-mode fibers with deep convolutional neural network
• Misalignment Resilient Diffractive Optical Networks
• RARE: Image Reconstruction using Deep Priors Learned without Ground Truth
• Artificial Neural Network with Physical Dynamic Metasurface Layer for Optimal Sensing
• Image transmission through a flexible multimode fiber by deep learning
• Single-Shot 3D Widefield Fluorescence Imaging with a Computational Miniature Mesoscope
• Learning-based real-time method to looking through scattering medium beyond the memory effect
• Intelligent Meta-Imagers: From Compressed to Learned Sensing
• Deep learning enabled laser speckle wavemeter with a high dynamic range
• Scale-, shift- and rotation-invariant diffractive optical networks
• Seeing through multimode fibers with real-valued intensity transmission matrices
• High-Resolution Limited-Angle Phase Tomography of Dense Layered Objects Using Deep Neural Networks
• All-optical image classification through unknown random diffusers using a single-pixel diffractive network
• Deep learning approach to coherent noise reduction in optical diffraction tomography
• Does deep learning always outperform simple linear regression in optical imaging?
• Deep Learning Enabled Real Time Speckle Recognition and Hyperspectral Imaging using a Multimode Fiber Array
• Data class-specific all-optical transformations and encryption
• Deep-learning-augmented Computational Miniature Mesoscope
• SIMBA: Scalable Inversion in Optical Tomography using Deep Denoising Priors
• A Two-step-training Deep Learning Framework for Real-time Computational Imaging without Physics Priors
• Displacement-agnostic coherent imaging through scatter with an interpretable deep neural network
• Adaptive 3D descattering with a dynamic synthesis network
• On the interplay between physical and content priors in deep learning for computational imaging
• Small-brain neural networks rapidly solve inverse problems with vortex Fourier encoders
• Embedded deep learning in ophthalmology: Making ophthalmic imaging smarter
• Deep learning-based photoacoustic imaging of vascular network through thick porous media
• Robust Position Sensing with Wave Fingerprints in Dynamic Complex Environments
• Ghost imaging based on Y-net: a dynamic coding and conjugate-decoding approach
• Spatial images from temporal data
• DeepLSR: a deep learning approach for laser speckle reduction
• Extracting particle size distribution from laser speckle with a physics-enhanced autocorrelation-based estimator (PEACE)
• Holographic particle localization under multiple scattering
• Neural-network-assisted in situ processing monitoring by speckle pattern observation
• Deep learning-enabled image quality control in tomographic reconstruction: Robust optical diffraction tomography
• Referenceless characterisation of complex media using physics-informed neural networks
• Recognizing three-dimensional phase images with deep learning
• Analysis of Diffractive Neural Networks for Seeing Through Random Diffusers
• Deep Learning Enhanced Quantum Holography with Undetected Photons
• Imaging trapped ion structures via fluorescence cross-correlation detection
• Deep learning-assisted imaging through stationary scattering media
• Hand Gestures Recognition in Videos Taken with Lensless Camera
• Diffuser-based computational imaging funduscope
• Beam Profiler Network (BPNet) -- A Deep Learning Approach to Mode Demultiplexing of Laguerre-Gaussian Optical Beams
• Deep Camera Obscura: An Image Restoration Pipeline for Lensless Pinhole Photography
• Assessing the properties of a colloidal suspension with the aid of deep learning
• Resampling the transmission matrix in an aberration-corrected Bessel mode basis
• Model-free estimation of the Cramér-Rao bound for deep-learning microscopy in complex media
• Fisher information flow in artificial neural networks
• Interleaved diffractive networks for information transfer through random diffusers
• Diffuse Optical Ptychography