Single chip photonic deep neural network with accelerated training

Abstract

As deep neural networks (DNNs) revolutionize machine learning, energy consumption and throughput are emerging as fundamental limitations of CMOS electronics. This has motivated a search for new hardware architectures optimized for artificial intelligence, such as electronic systolic arrays, memristor crossbar arrays, and optical accelerators. Optical systems can perform linear matrix operations at exceptionally high rate and efficiency, motivating recent demonstrations of low latency linear algebra and optical energy consumption below a photon per multiply-accumulate operation. However, demonstrating systems that co-integrate both linear and nonlinear processing units in a single chip remains a central challenge. Here we introduce such a system in a scalable photonic integrated circuit (PIC), enabled by several key advances: (i) high-bandwidth and low-power programmable nonlinear optical function units (NOFUs); (ii) coherent matrix multiplication units (CMXUs); and (iii) in situ training with optical acceleration. We experimentally demonstrate this fully-integrated coherent optical neural network (FICONN) architecture for a 3-layer DNN comprising 12 NOFUs and three CMXUs operating in the telecom C-band. Using in situ training on a vowel classification task, the FICONN achieves 92.7% accuracy on a test set, which is identical to the accuracy obtained on a digital computer with the same number of weights. This work lends experimental evidence to theoretical proposals for in situ training, unlocking orders of magnitude improvements in the throughput of training data. Moreover, the FICONN opens the path to inference at nanosecond latency and femtojoule per operation energy efficiency.

21 pages, 10 figures. Comments welcome

References in corpus (32)

• Deep Learning with Coherent Nanophotonic Circuits
• Parallel convolution processing using an integrated photonic tensor core
• 11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks
• Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems
• Single-chip photonic deep neural network for instantaneous image classification
• Attojoule Optoelectronics for Low-Energy Information Processing and Communications: a Tutorial Review
• Training of photonic neural networks through in situ backpropagation
• Self-Configuring Universal Linear Optical Component
• A one femtojoule athermal silicon modulator
• Bridging microcombs and silicon photonic engines for optoelectronics systems
• Fast inference of deep neural networks in FPGAs for particle physics
• Photonic Wire Bonds for Terabit/s Chip-to-Chip Interconnects
• Analysis Framework for the Prompt Discovery of Compact Binary Mergers in Gravitational-wave Data
• Reinforcement Learning in a large scale photonic Recurrent Neural Network
• A photonic integrated circuit based erbium-doped amplifier
• Reprogrammable Electro-Optic Nonlinear Activation Functions for Optical Neural Networks
• A silicon photonic modulator neuron
• Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks
• An optical neural network using less than 1 photon per multiplication
• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• Delocalized Photonic Deep Learning on the Internet's Edge
• Hardware error correction for programmable photonics
• Reconfigurable quantum photonics with on-chip detectors
• Matrix optimization on universal unitary photonic devices
• Measuring the Effects of Data Parallelism on Neural Network Training
• Self-learning photonic signal processor with an optical neural network chip
• Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs
• Enabling real-time multi-messenger astrophysics discoveries with deep learning
• Accurate Self-Configuration of Rectangular Multiport Interferometers
• Realizing a deep reinforcement learning agent discovering real-time feedback control strategies for a quantum system
• Stability of Self-Configuring Large Multiport Interferometers
• Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagation

Cited by in corpus (18)

• Training neural networks with end-to-end optical backpropagation
• All-optical nonlinear activation function based on stimulated Brillouin scattering
• Programmable metasurfaces for future photonic artificial intelligence
• Waveguide-multiplexed photonic matrix-vector multiplication processor using multiport photodetectors
• Asymmetrical estimator for training encapsulated deep photonic neural networks
• Nonlinear Inference Capacity of Fiber-Optical Extreme Learning Machines
• Hardware-Efficient Photonic Tensor Core: Accelerating Deep Neural Networks with Structured Compression
• Multi-timescale frequency-phase matching for high-yield nonlinear photonics
• High-precision programming of large-scale ring resonator circuits with minimal pre-calibration
• Arbitrary control over multimode wave propagation for machine learning
• Passive All-Optical Nonlinear Neuron Activation via PPLN Nanophotonic Waveguides
• Online training and pruning of multi-wavelength photonic neural networks
• Self-configuring high-speed multi-plane light conversion
• Quantum-secure multiparty deep learning
• Scaling of hardware-compatible perturbative training algorithms
• Effective programming of a photonic processor with complex interferometric structure
• Sub-quadratic scalable approximate linear converter using multi-plane light conversion with low-entropy mode mixers
• Mitigating quantum operation infidelity through engineering the distribution of photon losses