An optical neural network using less than 1 photon per multiplication

Abstract

Deep learning has rapidly become a widespread tool in both scientific and commercial endeavors. Milestones of deep learning exceeding human performance have been achieved for a growing number of tasks over the past several years, across areas as diverse as game-playing, natural-language translation, and medical-image analysis. However, continued progress is increasingly hampered by the high energy costs associated with training and running deep neural networks on electronic processors. Optical neural networks have attracted attention as an alternative physical platform for deep learning, as it has been theoretically predicted that they can fundamentally achieve higher energy efficiency than neural networks deployed on conventional digital computers. Here, we experimentally demonstrate an optical neural network achieving 99% accuracy on handwritten-digit classification using ~3.2 detected photons per weight multiplication and ~90% accuracy using ~0.64 photons (~$2.4\times 10^{-19}$ J of optical energy) per weight multiplication. This performance was achieved using a custom free-space optical processor that executes matrix-vector multiplications in a massively parallel fashion, with up to ~0.5 million scalar (weight) multiplications performed at the same time. Using commercially available optical components and standard neural-network training methods, we demonstrated that optical neural networks can operate near the standard quantum limit with extremely low optical powers and still achieve high accuracy. Our results provide a proof-of-principle for low-optical-power operation, and with careful system design including the surrounding electronics used for data storage and control, open up a path to realizing optical processors that require only $10^{-16}$ J total energy per scalar multiplication -- which is orders of magnitude more efficient than current digital processors.

42 pages, 21 figures

References in corpus (7)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Photonics for artificial intelligence and neuromorphic computing
• 11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks
• Waveguide-integrated black phosphorus photodetector with high responsivity and low dark current
• Attojoule Optoelectronics for Low-Energy Information Processing and Communications: a Tutorial Review
• Understanding and mitigating noise in trained deep neural networks
• Harnessing Optoelectronic Noises in a Photonic Generative Network

Cited by in corpus (37)

• The physics of optical computing
• Applications of integrated optical microcombs
• Intelligent Computing: The Latest Advances, Challenges and Future
• Image sensing with multilayer, nonlinear optical neural networks
• Delocalized Photonic Deep Learning on the Internet's Edge
• Neuromorphic computing using wavelength-division multiplexing
• Deep Learning with Coherent VCSEL Neural Networks
• Single chip photonic deep neural network with accelerated training
• Photonics for Neuromorphic Computing: Fundamentals, Devices, and Opportunities
• Inverse-designed low-index-contrast structures on silicon photonics platform for vector-matrix multiplication
• Quantum-limited stochastic optical neural networks operating at a few quanta per activation
• Ultrafast single-channel machine vision based on neuro-inspired photonic computing
• Information processing at the speed of light
• Characterization of multi-mode linear optical networks
• Incubating Advances in Integrated Photonics with Emerging Sensing and Computational Capabilities
• Waveguide-multiplexed photonic matrix-vector multiplication processor using multiport photodetectors
• Noise mitigation strategies in physical feedforward neural networks
• Ultrafast one-chip optical receiver with functional metasurface
• Practical Few-Atom Quantum Reservoir Computing
• AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks
• Predicting mode-locked fiber laser output using feed-forward neural network
• Impact of white noise in artificial neural networks trained for classification: performance and noise mitigation strategies
• Neural networks based on ultrafast time-delayed effects in exciton-polaritons
• SUANPAN: Scalable Photonic Linear Vector Machine
• The Physics of Learning
• Engineering nonlinear activation functions for all-optical neural networks via quantum interference
• Photonic systolic array for all-optical matrix-matrix multiplication
• Free-Space Optical Spiking Neural Network
• Polarimetric compressed sensing with hollow, self-assembled diffractive films
• Automating Experimental Optics with Sample Efficient Machine Learning Methods
• Harnessing Optoelectronic Noises in a Photonic Generative Network
• Requirements on bit resolution in optical Ising machine implementations
• Quantum-secure multiparty deep learning
• Dense Associative Memory in a Nonlinear Optical Hopfield Neural Network
• Noise impact on recurrent neural network with linear activation function
• Slimmed optical neural networks with multiplexed neuron sets and a corresponding backpropagation training algorithm
• All-fiber microendoscopic polarization sensing at single-photon level aided by deep-learning