Large-Scale Optical Neural Networks based on Photoelectric Multiplication

Abstract

Recent success in deep neural networks has generated strong interest in hardware accelerators to improve speed and energy consumption. This paper presents a new type of photonic accelerator based on coherent detection that is scalable to large ($N\gtrsim 10^6$) networks and can be operated at high (GHz) speeds and very low (sub-aJ) energies per multiply-and-accumulate (MAC), using the massive spatial multiplexing enabled by standard free-space optical components. In contrast to previous approaches, both weights and inputs are optically encoded so that the network can be reprogrammed and trained on the fly. Simulations of the network using models for digit- and image-classification reveal a "standard quantum limit" for optical neural networks, set by photodetector shot noise. This bound, which can be as low as 50 zJ/MAC, suggests performance below the thermodynamic (Landauer) limit for digital irreversible computation is theoretically possible in this device. The proposed accelerator can implement both fully-connected and convolutional networks. We also present a scheme for back-propagation and training that can be performed in the same hardware. This architecture will enable a new class of ultra-low-energy processors for deep learning.

arXiv commentText: 10 pages, 5 figures, 1 table. Supplementary: 8 pages, 5, figures, 2 tables

References in corpus (22)

• Improving neural networks by preventing co-adaptation of feature detectors
• Neural Message Passing for Quantum Chemistry
• Deep Learning with Coherent Nanophotonic Circuits
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Recent Trends in Deep Learning Based Natural Language Processing
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• cuDNN: Efficient Primitives for Deep Learning
• Neuromorphic Silicon Photonic Networks
• Deep Learning for Identifying Metastatic Breast Cancer
• Optoelectronic Reservoir Computing
• Attojoule Optoelectronics for Low-Energy Information Processing and Communications: a Tutorial Review
• Ultrahigh-efficiency second-harmonic generation in nanophotonic PPLN waveguides
• Breaking the Loss Limitation of On-chip High-confinement Resonators
• Network of Time-Multiplexed Optical Parametric Oscillators as a Coherent Ising Machine
• A one femtojoule athermal silicon modulator
• First Long-Term Application of Squeezed States of Light in a Gravitational-Wave Observatory
• Large-scale Ising spin network based on degenerate optical parametric oscillators
• Quantum limits in interferometric measurements
• Inverse Design of Near Unity Efficiency Perfectly Vertical Grating Couplers
• On-Chip Optical Convolutional Neural Networks
• Remnants of semiclassical bistability in the few-photon regime of cavity QED
• Silicon Optical Phased Array with High-Efficiency Beam Formation over 180 Degree Field of View

Cited by in corpus (74)

• Integrated photonics on thin-film lithium niobate
• Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
• Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks
• An optical neural network using less than 1 photon per multiplication
• Machine-Learning-Assisted Metasurface Design for High-Efficiency Thermal Emitter Optimization
• High-speed programmable photonic circuits in a cryogenically compatible, visible-NIR 200 mm CMOS architecture
• Delocalized Photonic Deep Learning on the Internet's Edge
• Neuromorphic computing using wavelength-division multiplexing
• Heuristic Recurrent Algorithms for Photonic Ising Machines
• Optical coherent dot-product chip for sophisticated deep learning regression
• Polaritonic neuromorphic computing outperforms linear classifiers
• 9~GHz measurement of squeezed light by interfacing silicon photonics and integrated electronics
• Photonic extreme learning machine by free-space optical propagation
• Applications and Techniques for Fast Machine Learning in Science
• Noise-enhanced spatial-photonic Ising machine
• Silicon photodetector for integrated lithium niobate photonics
• Scalable spin-glass optical simulator
• Photonic neural networks based on integrated silicon microresonators
• Accurate Self-Configuration of Rectangular Multiport Interferometers
• Biasing the quantum vacuum to control macroscopic probability distributions
• Piezo-optomechanical cantilever modulators for VLSI visible photonics
• High-Speed Phase-Only Spatial Light Modulators with Two-Dimensional Tunable Microcavity Arrays
• Quantifying power use in silicon photonic neural networks
• Cascadable all-optical NAND gates using diffractive networks
• Deep Photonic Reservoir Computer Based on Frequency Multiplexing with Fully Analog Connection Between Layers
• Scaling optical computing in synthetic frequency dimension using integrated cavity acousto-optics
• Stability of Self-Configuring Large Multiport Interferometers
• Developing of a photonic hardware platform for brain-inspired computing based on $5\times 5$ VCSEL arrays
• Hardware realization of the multiply and accumulate operation on radio-frequency signals with magnetic tunnel junctions
• Fock State-enhanced Expressivity of Quantum Machine Learning Models
• Optoelectronic Intelligence
• Characterizing Coherent Integrated Photonic Neural Networks under Imperfections
• Deep-Learning-Enabled Inverse Engineering of Multi-Wavelength Invisibility-to-Superscattering Switching with Phase-Change Materials
• Wave-based extreme deep learning based on non-linear time-Floquet entanglement
• Complex-valued universal linear transformations and image encryption using spatially incoherent diffractive networks
• Architecture agnostic algorithm for reconfigurable optical interferometer programming
• Robust and Efficient Single-Pixel Image Classificationwith Nonlinear Optics
• Single-Pixel Pattern Recognition with Coherent Nonlinear Optics
• Ultra-broadband surface-normal coherent optical receiver with nanometallic polarizers
• Optical Neural Network Based on Synthetic Nonlinear Photonic Lattices
• Reconfigurable nonlinear optical element using tunable couplers and inverse-designed structure
• Waveguide-multiplexed photonic matrix-vector multiplication processor using multiport photodetectors
• Integration of Programmable Diffraction with Digital Neural Networks
• Decision-making and control with diffractive optical networks
• AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks
• A Non-Volatile All-Spin Non-Binary Matrix Multiplier: An Efficient Hardware Accelerator for Machine Learning
• Fast reconstruction of programmable integrated interferometers
• Image classification using collective optical modes of an array of nanolasers
• Optical neural network architecture for deep learning with the temporal synthetic dimension
• Slow and fast topological dynamical phase transitions in a Duffing resonator driven by two detuned tones
• Straintronic magnetic tunnel junctions for analog computation: A perspective
• A 262 TOPS Hyperdimensional Photonic AI Accelerator powered by a Si3N4 microcomb laser
• Engineering nonlinear activation functions for all-optical neural networks via quantum interference
• Photonic neural field on a silicon chip: large-scale, high-speed neuro-inspired computing and sensing
• Photonic systolic array for all-optical matrix-matrix multiplication
• Scalable intensity-based photonic matrix-vector multiplication processor using single-wavelength time-division-multiplexed signals
• Demonstration of Hardware Efficient Photonic Variational Quantum Algorithm
• Reversible ternary logic with Laguerre-Gaussian modes
• Quantum Optical Convolutional Neural Network: A Novel Image Recognition Framework for Quantum Computing
• Adiabatic evolution on a spatial-photonic Ising machine
• Quantum-secure multiparty deep learning
• Performance evaluation of an integrated photonic convolutional neural network based on delay buffering and wavelength division multiplexing
• Harnessing Optoelectronic Noises in a Photonic Generative Network
• Compute-first optical detection for noise-resilient visual perception
• Unitary Learning for Deep Diffractive Neural Network
• Shot-Noise Limited Homodyne Detection for MHz Quantum Light Characterisation in the 2 μm Band
• A Diffractive Neural Network with Weight-Noise-Injection Training
• Real-time image processing with a 2D semiconductor neural network vision sensor
• Training of mixed-signal optical convolutional neural network with reduced quantization level
• End-to-End Framework for Efficient Deep Learning Using Metasurfaces Optics
• Neural networks with quantum states of light
• Compatible Learning for Deep Photonic Neural Network
• Implementation of transformer-based LLMs with large-scale optoelectronic neurons on a CMOS compatible platform
• Implementation of Optical Deep Neural Networks using the Fabry-Perot Interferometer