Scale-, shift- and rotation-invariant diffractive optical networks

Abstract

Recent research efforts in optical computing have gravitated towards developing optical neural networks that aim to benefit from the processing speed and parallelism of optics/photonics in machine learning applications. Among these endeavors, Diffractive Deep Neural Networks (D2NNs) harness light-matter interaction over a series of trainable surfaces, designed using deep learning, to compute a desired statistical inference task as the light waves propagate from the input plane to the output field-of-view. Although, earlier studies have demonstrated the generalization capability of diffractive optical networks to unseen data, achieving e.g., >98% image classification accuracy for handwritten digits, these previous designs are in general sensitive to the spatial scaling, translation and rotation of the input objects. Here, we demonstrate a new training strategy for diffractive networks that introduces input object translation, rotation and/or scaling during the training phase as uniformly distributed random variables to build resilience in their blind inference performance against such object transformations. This training strategy successfully guides the evolution of the diffractive optical network design towards a solution that is scale-, shift- and rotation-invariant, which is especially important and useful for dynamic machine vision applications in e.g., autonomous cars, in-vivo imaging of biomedical specimen, among others.

arXiv comment28 Pages, 6 Figures, 1 Table

References in corpus (22)

• Deep Learning with Coherent Nanophotonic Circuits
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Training deep neural networks for the inverse design of nanophotonic structures
• Neuromorphic Silicon Photonic Networks
• Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue
• Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media
• Training of photonic neural networks through in situ backpropagation
• PhaseStain: Digital staining of label-free quantitative phase microscopy images using deep learning
• Reinforcement Learning in a large scale photonic Recurrent Neural Network
• Wave Physics as an Analog Recurrent Neural Network
• Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery
• Design of Task-Specific Optical Systems Using Broadband Diffractive Neural Networks
• Analysis of Diffractive Optical Neural Networks and Their Integration with Electronic Neural Networks
• Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning
• Class-specific Differential Detection in Diffractive Optical Neural Networks Improves Inference Accuracy
• Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue
• Misalignment Resilient Diffractive Optical Networks
• Ensemble learning of diffractive optical networks
• Cross-modality deep learning brings bright-field microscopy contrast to holography
• Neuromorphic photonics with electro-absorption modulators
• PCNNA: A Photonic Convolutional Neural Network Accelerator
• Principles of Neuromorphic Photonics

Cited by in corpus (18)

• Deep learning in nano-photonics: inverse design and beyond
• Spectrally-Encoded Single-Pixel Machine Vision Using Diffractive Networks
• Polarization Multiplexed Diffractive Computing: All-Optical Implementation of a Group of Linear Transformations Through a Polarization-Encoded Diffractive Network
• Massively Parallel Universal Linear Transformations using a Wavelength-Multiplexed Diffractive Optical Network
• Rapid Sensing of Hidden Objects and Defects using a Single-Pixel Diffractive Terahertz Processor
• All-optical image denoising using a diffractive visual processor
• Cascadable all-optical NAND gates using diffractive networks
• Universal Polarization Transformations: Spatial programming of polarization scattering matrices using a deep learning-designed diffractive polarization transformer
• Classification and reconstruction of spatially overlapping phase images using diffractive optical networks
• Simultaneously sorting vector vortex beams of 120 modes
• Ultrahigh-fidelity spatial mode quantum gates in high-dimensional space by diffractive deep neural networks
• Complex-valued universal linear transformations and image encryption using spatially incoherent diffractive networks
• Multiplexed all-optical permutation operations using a reconfigurable diffractive optical network
• Optoelectronic generative adversarial networks
• Coherence Awareness in Diffractive Neural Networks
• Computer-free, all-optical reconstruction of holograms using diffractive networks
• Snapshot 3D image projection using a diffractive decoder
• All-Optical Synthesis of an Arbitrary Linear Transformation Using Diffractive Surfaces