Integration of Programmable Diffraction with Digital Neural Networks
arXiv:2406.10688 · doi:10.1021/acsphotonics.4c01099
Abstract
Optical imaging and sensing systems based on diffractive elements have seen massive advances over the last several decades. Earlier generations of diffractive optical processors were, in general, designed to deliver information to an independent system that was separately optimized, primarily driven by human vision or perception. With the recent advances in deep learning and digital neural networks, there have been efforts to establish diffractive processors that are jointly optimized with digital neural networks serving as their back-end. These jointly optimized hybrid (optical+digital) processors establish a new "diffractive language" between input electromagnetic waves that carry analog information and neural networks that process the digitized information at the back-end, providing the best of both worlds. Such hybrid designs can process spatially and temporally coherent, partially coherent, or incoherent input waves, providing universal coverage for any spatially varying set of point spread functions that can be optimized for a given task, executed in collaboration with digital neural networks. In this article, we highlight the utility of this exciting collaboration between engineered and programmed diffraction and digital neural networks for a diverse range of applications. We survey some of the major innovations enabled by the push-pull relationship between analog wave processing and digital neural networks, also covering the significant benefits that could be reaped through the synergy between these two complementary paradigms.
arXiv comment30 Pages, 6 Figures
References in corpus (58)
- Deep Learning with Coherent Nanophotonic Circuits
- All-Optical Machine Learning Using Diffractive Deep Neural Networks
- 11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks
- Training deep neural networks for the inverse design of nanophotonic structures
- A Generative Model for Inverse Design of Metamaterials
- Rapid identification of pathogenic bacteria using Raman spectroscopy and deep learning
- Neuromorphic Silicon Photonic Networks
- Art and the science of generative AI: A deeper dive
- Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit
- Deep neural networks for the evaluation and design of photonic devices
- Training of photonic neural networks through in situ backpropagation
- Deep learning in nano-photonics: inverse design and beyond
- Freeform Diffractive Metagrating Design Based on Generative Adversarial Networks
- Large-Scale Optical Neural Networks based on Photoelectric Multiplication
- Metasurface Optics for Full-color Computational Imaging
- Deep Learning-enabled Virtual Histological Staining of Biological Samples
- DeepSTORM3D: dense three dimensional localization microscopy and point spread function design by deep learning
- Design of Task-Specific Optical Systems Using Broadband Diffractive Neural Networks
- All-dielectric phase-change reconfigurable metasurface
- Deep learning approach to Fourier ptychographic microscopy
- Analysis of Diffractive Optical Neural Networks and Their Integration with Electronic Neural Networks
- Digital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs)
- Spectrally-Encoded Single-Pixel Machine Vision Using Diffractive Networks
- Terahertz Pulse Shaping Using Diffractive Surfaces
- Computational Imaging Without a Computer: Seeing Through Random Diffusers at the Speed of Light
- All-Optical Information Processing Capacity of Diffractive Surfaces
- Misalignment Resilient Diffractive Optical Networks
- Artificial Neural Network with Physical Dynamic Metasurface Layer for Optimal Sensing
- Automated screening of sickle cells using a smartphone-based microscope and deep learning
- Snapshot Multispectral Imaging Using a Diffractive Optical Network
- Diffractive all-optical computing for quantitative phase imaging
- Massively Parallel Universal Linear Transformations using a Wavelength-Multiplexed Diffractive Optical Network
- Neural network-based on-chip spectroscopy using a scalable plasmonic encoder
- Feature visualization of Raman spectrum analysis with deep convolutional neural network
- To image, or not to image: Class-specific diffractive cameras with all-optical erasure of undesired objects
- Deep learning in biomedical optics
- Rapid Sensing of Hidden Objects and Defects using a Single-Pixel Diffractive Terahertz Processor
- Universal Linear Intensity Transformations Using Spatially-Incoherent Diffractive Processors
- Illumination Pattern Design with Deep Learning for Single-Shot Fourier Ptychographic Microscopy
- Super-resolution image display using diffractive decoders
- Data class-specific all-optical transformations and encryption
- Unidirectional Imaging using Deep Learning-Designed Materials
- Deep learning-enabled multiplexed point-of-care sensor using a paper-based fluorescence vertical flow assay
- All-optical complex field imaging using diffractive processors
- Neural network-based image reconstruction in swept-source optical coherence tomography using undersampled spectral data
- Non-imaging single-pixel sensing with optimized binary modulation
- Universal Polarization Transformations: Spatial programming of polarization scattering matrices using a deep learning-designed diffractive polarization transformer
- Information hiding cameras: optical concealment of object information into ordinary images
- Learning Diffractive Optical Communication Around Arbitrary Opaque Occlusions
- Pyramid diffractive optical networks for unidirectional image magnification and demagnification
- Multispectral Quantitative Phase Imaging Using a Diffractive Optical Network
- Broadband nonlinear modulation of incoherent light using a transparent optoelectronic neuron array
- Time-lapse image classification using a diffractive neural network
- All-Optical Phase Conjugation Using Diffractive Wavefront Processing
- Optical information transfer through random unknown diffusers using electronic encoding and diffractive decoding
- Analysis of Diffractive Neural Networks for Seeing Through Random Diffusers
- DUF: Deep Coded Aperture Design and Unrolling Algorithm for Compressive Spectral Image Fusion
- Compressed Meta-Optical Encoder for Image Classification