Integration of Programmable Diffraction with Digital Neural Networks

Abstract

Optical imaging and sensing systems based on diffractive elements have seen massive advances over the last several decades. Earlier generations of diffractive optical processors were, in general, designed to deliver information to an independent system that was separately optimized, primarily driven by human vision or perception. With the recent advances in deep learning and digital neural networks, there have been efforts to establish diffractive processors that are jointly optimized with digital neural networks serving as their back-end. These jointly optimized hybrid (optical+digital) processors establish a new "diffractive language" between input electromagnetic waves that carry analog information and neural networks that process the digitized information at the back-end, providing the best of both worlds. Such hybrid designs can process spatially and temporally coherent, partially coherent, or incoherent input waves, providing universal coverage for any spatially varying set of point spread functions that can be optimized for a given task, executed in collaboration with digital neural networks. In this article, we highlight the utility of this exciting collaboration between engineered and programmed diffraction and digital neural networks for a diverse range of applications. We survey some of the major innovations enabled by the push-pull relationship between analog wave processing and digital neural networks, also covering the significant benefits that could be reaped through the synergy between these two complementary paradigms.

30 Pages, 6 Figures

References in corpus (9)

• 11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks
• Misalignment Resilient Diffractive Optical Networks
• Diffractive all-optical computing for quantitative phase imaging
• Neural network-based on-chip spectroscopy using a scalable plasmonic encoder
• Feature visualization of Raman spectrum analysis with deep convolutional neural network
• Deep learning in biomedical optics
• Deep learning-enabled multiplexed point-of-care sensor using a paper-based fluorescence vertical flow assay
• Multispectral Quantitative Phase Imaging Using a Diffractive Optical Network
• Compressed Meta-Optical Encoder for Image Classification