Misalignment Resilient Diffractive Optical Networks

Abstract

As an optical machine learning framework, Diffractive Deep Neural Networks (D2NN) take advantage of data-driven training methods used in deep learning to devise light-matter interaction in 3D for performing a desired statistical inference task. Multi-layer optical object recognition platforms designed with this diffractive framework have been shown to generalize to unseen image data achieving e.g., >98% blind inference accuracy for hand-written digit classification. The multi-layer structure of diffractive networks offers significant advantages in terms of their diffraction efficiency, inference capability and optical signal contrast. However, the use of multiple diffractive layers also brings practical challenges for the fabrication and alignment of these diffractive systems for accurate optical inference. Here, we introduce and experimentally demonstrate a new training scheme that significantly increases the robustness of diffractive networks against 3D misalignments and fabrication tolerances in the physical implementation of a trained diffractive network. By modeling the undesired layer-to-layer misalignments in 3D as continuous random variables in the optical forward model, diffractive networks are trained to maintain their inference accuracy over a large range of misalignments; we term this diffractive network design as vaccinated D2NN (v-D2NN). We further extend this vaccination strategy to the training of diffractive networks that use differential detectors at the output plane as well as to jointly-trained hybrid (optical-electronic) networks to reveal that all of these diffractive designs improve their resilience to misalignments by taking into account possible 3D fabrication variations and displacements during their training phase.

15 Pages, 6 Figures

References in corpus (28)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Deep Learning with Coherent Nanophotonic Circuits
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Training deep neural networks for the inverse design of nanophotonic structures
• Phase recovery and holographic image reconstruction using deep learning in neural networks
• Neuromorphic Silicon Photonic Networks
• Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue
• Deep Learning Microscopy
• Deep-STORM: super-resolution single-molecule microscopy by deep learning
• Network of Time-Multiplexed Optical Parametric Oscillators as a Coherent Ising Machine
• Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media
• PhaseStain: Digital staining of label-free quantitative phase microscopy images using deep learning
• Reinforcement Learning in a large scale photonic Recurrent Neural Network
• Wave Physics as an Analog Recurrent Neural Network
• Metasurface interferometry towards quantum sensors
• Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery
• Design of Task-Specific Optical Systems Using Broadband Diffractive Neural Networks
• Analysis of Diffractive Optical Neural Networks and Their Integration with Electronic Neural Networks
• Digital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs)
• Class-specific Differential Detection in Diffractive Optical Neural Networks Improves Inference Accuracy
• Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue
• Cross-modality deep learning brings bright-field microscopy contrast to holography
• Neuromorphic photonics with electro-absorption modulators
• Measurement of Anomalous Diffusion Using Recurrent Neural Networks
• PCNNA: A Photonic Convolutional Neural Network Accelerator
• Coupled nonlinear delay systems as deep convolutional neural networks
• Principles of Neuromorphic Photonics

Cited by in corpus (50)

• Spectrally-Encoded Single-Pixel Machine Vision Using Diffractive Networks
• All-Optical Information Processing Capacity of Diffractive Surfaces
• Ensemble learning of diffractive optical networks
• Polarization Multiplexed Diffractive Computing: All-Optical Implementation of a Group of Linear Transformations Through a Polarization-Encoded Diffractive Network
• Snapshot Multispectral Imaging Using a Diffractive Optical Network
• Diffractive all-optical computing for quantitative phase imaging
• Massively Parallel Universal Linear Transformations using a Wavelength-Multiplexed Diffractive Optical Network
• To image, or not to image: Class-specific diffractive cameras with all-optical erasure of undesired objects
• Scale-, shift- and rotation-invariant diffractive optical networks
• Rapid Sensing of Hidden Objects and Defects using a Single-Pixel Diffractive Terahertz Processor
• All-optical image classification through unknown random diffusers using a single-pixel diffractive network
• Universal Linear Intensity Transformations Using Spatially-Incoherent Diffractive Processors
• Super-resolution image display using diffractive decoders
• All-optical image denoising using a diffractive visual processor
• Data class-specific all-optical transformations and encryption
• Unidirectional Imaging using Deep Learning-Designed Materials
• Multiplane Quantitative Phase Imaging Using a Wavelength-Multiplexed Diffractive Optical Processor
• All-optical complex field imaging using diffractive processors
• Optical Generative Models
• Cascadable all-optical NAND gates using diffractive networks
• Universal Polarization Transformations: Spatial programming of polarization scattering matrices using a deep learning-designed diffractive polarization transformer
• Classification and reconstruction of spatially overlapping phase images using diffractive optical networks
• Information hiding cameras: optical concealment of object information into ordinary images
• Quantitative phase imaging (QPI) through random diffusers using a diffractive optical network
• Ultrahigh-fidelity spatial mode quantum gates in high-dimensional space by diffractive deep neural networks
• Subwavelength Imaging using a Solid-Immersion Diffractive Optical Processor
• Multispectral Quantitative Phase Imaging Using a Diffractive Optical Network
• Complex-valued universal linear transformations and image encryption using spatially incoherent diffractive networks
• Time-lapse image classification using a diffractive neural network
• All-Optical Phase Conjugation Using Diffractive Wavefront Processing
• Optical information transfer through random unknown diffusers using electronic encoding and diffractive decoding
• Multiplexed all-optical permutation operations using a reconfigurable diffractive optical network
• Analysis of Diffractive Neural Networks for Seeing Through Random Diffusers
• Machine-Learning-Assisted Photonic Device Development: A Multiscale Approach from Theory to Characterization
• Integration of Programmable Diffraction with Digital Neural Networks
• Universal point spread function engineering for 3D optical information processing
• Polarization and Orbital Angular Momentum Encoded Quantum Toffoli Gate Enabled by Diffractive Neural Networks
• Lossless, Complex-Valued Optical Field Control with Compound Metaoptics
• Massively parallel and universal approximation of nonlinear functions using diffractive processors
• Broadband Unidirectional Visible Imaging Using Wafer-Scale Nano-Fabrication of Multi-Layer Diffractive Optical Processors
• Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization
• Optimizing Structured Surfaces for Diffractive Waveguides
• Coherence Awareness in Diffractive Neural Networks
• Unidirectional focusing of light using structured diffractive surfaces
• Harnessing Optoelectronic Noises in a Photonic Generative Network
• Compute-first optical detection for noise-resilient visual perception
• Computer-free, all-optical reconstruction of holograms using diffractive networks
• Diffractive Interconnects: All-Optical Permutation Operation Using Diffractive Networks
• Snapshot 3D image projection using a diffractive decoder
• All-Optical Synthesis of an Arbitrary Linear Transformation Using Diffractive Surfaces