An Optical Frontend for a Convolutional Neural Network

Abstract

The parallelism of optics and the miniaturization of optical components using nanophotonic structures, such as metasurfaces present a compelling alternative to electronic implementations of convolutional neural networks. The lack of a low-power optical nonlinearity, however, requires slow and energy-inefficient conversions between the electronic and optical domains. Here, we design an architecture which utilizes a single electrical to optical conversion by designing a free-space optical frontend unit that implements the linear operations of the first layer with the subsequent layers realized electronically. Speed and power analysis of the architecture indicates that the hybrid photonic-electronic architecture outperforms sole electronic architecture for large image sizes and kernels. Benchmarking of the photonic-electronic architecture on a modified version of AlexNet achieves a classification accuracy of 87% on images from the Kaggle Cats and Dogs challenge database.

References in corpus (11)

• Deep Learning in Neural Networks: An Overview
• Deep Learning with Coherent Nanophotonic Circuits
• Dielectric Metasurfaces for Complete Control of Phase and Polarization with Subwavelength Spatial Resolution and High Transmission
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Training of photonic neural networks through in situ backpropagation
• Ultrafast all-optical switching by single photons
• Reinforcement Learning in a large scale photonic Recurrent Neural Network
• Low Contrast Dielectric Metasurface Optics
• Large Area Metalenses: Design, Characterization, and Mass Manufacturing
• Varifocal zoom imaging with large area focal length adjustable metalenses
• Ultrafast photon-photon interaction in a strongly coupled quantum dot-cavity system

Cited by in corpus (16)

• Parallel convolution processing using an integrated photonic tensor core
• The physics of optical computing
• Reprogrammable Electro-Optic Nonlinear Activation Functions for Optical Neural Networks
• Image sensing with multilayer, nonlinear optical neural networks
• Meta-optics for spatial optical analog computing
• Free-space optical neural network based on thermal atomic nonlinearity
• Photonic convolutional neural networks using integrated diffractive optics
• Universal metasurfaces for complete linear control of coherent light transmission
• Classification and reconstruction of spatially overlapping phase images using diffractive optical networks
• Knowledge Distillation Circumvents Nonlinearity for Optical Convolutional Neural Networks
• Programmable metasurfaces for future photonic artificial intelligence
• Integrated Photonic Fractional Convolution Accelerator
• Image amplification in a self-imaging degenerate optical cavity
• Quantum optical classifier with superexponential speedup
• High speed and reconfigurable optronic neural network with digital nonlinear activation
• A neuromorphic camera for tracking passive and active matter with lower data throughput