Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks

Abstract

Neural networks are widely deployed models across many scientific disciplines and commercial endeavors ranging from edge computing and sensing to large-scale signal processing in data centers. The most efficient and well-entrenched method to train such networks is backpropagation, or reverse-mode automatic differentiation. To counter an exponentially increasing energy budget in the artificial intelligence sector, there has been recent interest in analog implementations of neural networks, specifically nanophotonic neural networks for which no analog backpropagation demonstration exists. We design mass-manufacturable silicon photonic neural networks that alternately cascade our custom designed "photonic mesh" accelerator with digitally implemented nonlinearities. These reconfigurable photonic meshes program computationally intensive arbitrary matrix multiplication by setting physical voltages that tune the interference of optically encoded input data propagating through integrated Mach-Zehnder interferometer networks. Here, using our packaged photonic chip, we demonstrate in situ backpropagation for the first time to solve classification tasks and evaluate a new protocol to keep the entire gradient measurement and update of physical device voltages in the analog domain, improving on past theoretical proposals. Our method is made possible by introducing three changes to typical photonic meshes: (1) measurements at optical "grating tap" monitors, (2) bidirectional optical signal propagation automated by fiber switch, and (3) universal generation and readout of optical amplitude and phase. After training, our classification achieves accuracies similar to digital equivalents even in presence of systematic error. Our findings suggest a new training paradigm for photonics-accelerated artificial intelligence based entirely on a physical analog of the popular backpropagation technique.

23 pages, 10 figures

References in corpus (5)

• Attojoule Optoelectronics for Low-Energy Information Processing and Communications: a Tutorial Review
• Efficient, Compact and Low Loss Thermo-Optic Phase Shifter in Silicon
• Compute Trends Across Three Eras of Machine Learning
• Direct Feedback Alignment Provides Learning in Deep Neural Networks
• MEMS for Photonic Integrated Circuits

Cited by in corpus (31)

• Single chip photonic deep neural network with accelerated training
• Photonics for Neuromorphic Computing: Fundamentals, Devices, and Opportunities
• Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation
• Biasing the quantum vacuum to control macroscopic probability distributions
• Chip-to-chip optical multimode communication with universal mode processors
• Measuring, processing, and generating partially coherent light with self-configuring optics
• Training neural networks with end-to-end optical backpropagation
• Tackling Sampling Noise in Physical Systems for Machine Learning Applications: Fundamental Limits and Eigentasks
• Symmetric silicon microring resonator optical crossbar array for accelerated inference and training in deep learning
• Multiplexed all-optical permutation operations using a reconfigurable diffractive optical network
• Programmable metasurfaces for future photonic artificial intelligence
• Training Coupled Phase Oscillators as a Neuromorphic Platform using Equilibrium Propagation
• Mirage: An RNS-Based Photonic Accelerator for DNN Training
• Photonic integrated processor for structured light detection and distinction
• Asymmetrical estimator for training encapsulated deep photonic neural networks
• Unitary control of partially coherent waves. II. Transmission or reflection
• Programmable photonic time circuits for highly scalable universal unitaries
• A Blueprint for Precise and Fault-Tolerant Analog Neural Networks
• Inverse design and experimental realization of plasma metamaterials
• Physical Neural Networks with Self-Learning Capabilities
• Arbitrary control over multimode wave propagation for machine learning
• Low-depth, compact and error-tolerant photonic matrix-vector multiplication beyond the unitary group
• SUANPAN: Scalable Photonic Linear Vector Machine
• Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization
• Nonlinear unitary circuits for photonic neural networks
• Passive All-Optical Nonlinear Neuron Activation via PPLN Nanophotonic Waveguides
• Quantum-secure multiparty deep learning
• Online training and pruning of multi-wavelength photonic neural networks
• Scaling of hardware-compatible perturbative training algorithms
• Polaritonic Machine Learning for Graph-based Data Analysis
• A ghost mechanism: An analytical model of abrupt learning in recurrent networks