Training of photonic neural networks through in situ backpropagation

Abstract

Recently, integrated optics has gained interest as a hardware platform for implementing machine learning algorithms. Of particular interest are artificial neural networks, since matrix-vector multi- plications, which are used heavily in artificial neural networks, can be done efficiently in photonic circuits. The training of an artificial neural network is a crucial step in its application. However, currently on the integrated photonics platform there is no efficient protocol for the training of these networks. In this work, we introduce a method that enables highly efficient, in situ training of a photonic neural network. We use adjoint variable methods to derive the photonic analogue of the backpropagation algorithm, which is the standard method for computing gradients of conventional neural networks. We further show how these gradients may be obtained exactly by performing intensity measurements within the device. As an application, we demonstrate the training of a numerically simulated photonic artificial neural network. Beyond the training of photonic machine learning implementations, our method may also be of broad interest to experimental sensitivity analysis of photonic systems and the optimization of reconfigurable optics platforms.

12 pages, 6 figures

References in corpus (10)

• Deep Learning with Coherent Nanophotonic Circuits
• Training and Operation of an Integrated Neuromorphic Network Based on Metal-Oxide Memristors
• Universal Linear Optics
• Quantum transport simulations in a programmable nanophotonic processor
• Self-Configuring Universal Linear Optical Component
• All-optical mode unscrambling on a silicon photonic chip
• Superconducting optoelectronic circuits for neuromorphic computing
• Self-aligning universal beam coupler
• Trainable and Dynamic Computing: Error Backpropagation through Physical Media
• Method for Computationally Efficient Design of Dielectric Laser Accelerators

Cited by in corpus (87)

• Photonics for artificial intelligence and neuromorphic computing
• Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems
• Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit
• Laguerre-Gaussian mode sorter
• All Optical Neural Network with Nonlinear Activation Functions
• Wave Physics as an Analog Recurrent Neural Network
• Reprogrammable Electro-Optic Nonlinear Activation Functions for Optical Neural Networks
• Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks
• Intelligent Nanophotonics: Merging Photonics and Artificial Intelligence at the Nanoscale
• Analysis of Diffractive Optical Neural Networks and Their Integration with Electronic Neural Networks
• Digital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs)
• Machine-Learning-Assisted Metasurface Design for High-Efficiency Thermal Emitter Optimization
• Single chip photonic deep neural network with accelerated training
• Neuromorphic computing using wavelength-division multiplexing
• Prospects and applications of photonic neural networks
• Hardware error correction for programmable photonics
• Class-specific Differential Detection in Diffractive Optical Neural Networks Improves Inference Accuracy
• Intelligent metaphotonics empowered by machine learning
• In-memory photonic dot-product engine with electrically programmable weight banks
• Silicon microring synapses enable photonic deep learning beyond 9-bit precision
• Matrix optimization on universal unitary photonic devices
• Ensemble learning of diffractive optical networks
• Self-learning photonic signal processor with an optical neural network chip
• Three dimensional waveguide-interconnects for scalable integration of photonic neural networks
• 8x8 Reconfigurable quantum photonic processor based on silicon nitride waveguides
• Optical coherent dot-product chip for sophisticated deep learning regression
• An Optical Frontend for a Convolutional Neural Network
• Photonics for Neuromorphic Computing: Fundamentals, Devices, and Opportunities
• Computational inverse design for cascaded systems of metasurface optics
• Photonic extreme learning machine by free-space optical propagation
• Robust architecture for programmable universal unitaries
• Nonlinear optical encoding enabled by recurrent linear scattering
• Silicon Photonic Architecture for Training Deep Neural Networks with Direct Feedback Alignment
• Scaling Up Silicon Photonic-based Accelerators: Challenges and Opportunities
• Forward-Mode Differentiation of Maxwell's Equations
• Scale-, shift- and rotation-invariant diffractive optical networks
• Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation
• Efficient training and design of photonic neural network through neuroevolution
• Asymptotically Fault-Tolerant Programmable Photonics
• All-Optical Spiking Neuron Based On Passive Micro-Resonator
• Accurate Self-Configuration of Rectangular Multiport Interferometers
• Coherent photonic crossbar as a universal linear operator
• Optimal design of error-tolerant reprogrammable multiport interferometers
• Rapid classification of quantum sources enabled by machine learning
• Machine learning and evolutionary algorithm studies of graphene metamaterials for optimized plasmon-induced transparency
• Photonic convolutional neural networks using integrated diffractive optics
• Addressing the programming challenges of practical interferometric mesh based optical processors
• Stability of Self-Configuring Large Multiport Interferometers
• Scalable and Robust Photonic Integrated Unitary Converter
• Measuring, processing, and generating partially coherent light with self-configuring optics
• Neural Schrödinger Equation:Physical Law as Neural Network
• Photonic architecture for reinforcement learning
• Optoelectronic Intelligence
• Self-learning Machines based on Hamiltonian Echo Backpropagation
• Universal programmable photonic architecture for quantum information processing
• A photonic complex perceptron for ultrafast data processing
• Training neural networks with end-to-end optical backpropagation
• Broadband nonlinear modulation of incoherent light using a transparent optoelectronic neuron array
• Architecture agnostic algorithm for reconfigurable optical interferometer programming
• Reconfigurable Photonic Circuit for Controlled Power Delivery to Laser-Driven Accelerators on a Chip
• Topological Bulk Lasing Modes Using an Imaginary Gauge Field
• Symmetric silicon microring resonator optical crossbar array for accelerated inference and training in deep learning
• Ultra-high endurance silicon photonic memory using vanadium dioxide
• Optical Neural Network Based on Synthetic Nonlinear Photonic Lattices
• Training Coupled Phase Oscillators as a Neuromorphic Platform using Equilibrium Propagation
• The multimode four-photon Hong-Ou-Mandel interference
• Integration of Programmable Diffraction with Digital Neural Networks
• Mirage: An RNS-Based Photonic Accelerator for DNN Training
• Asymmetrical estimator for training encapsulated deep photonic neural networks
• Optical Phase Dropout in Diffractive Deep Neural Network
• On-chip Optical Phase Monitoring in Multi-Transverse-Mode Integrated Silicon-based Optical Processors
• High-performance real-world optical computing trained by in situ gradient-based model-free optimization
• Express Wavenet -- a low parameter optical neural network with random shift wavelet pattern
• Image classification using collective optical modes of an array of nanolasers
• Fast reconstruction of programmable integrated interferometers
• Optimal Data Generation in Multi-Dimensional Parameter Spaces, using Bayesian Optimization
• Physical Neural Networks with Self-Learning Capabilities
• Broadband optical switch based on an achromatic photonic gauge potential in dynamically modulated waveguides
• Optical Neural Ordinary Differential Equations
• Optical neural network architecture for deep learning with the temporal synthetic dimension
• Inverse design of plasma metamaterial devices with realistic elements
• Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization
• Fast reconstruction of programmable interferometers with intensity-only measurements
• Engineering nonlinear activation functions for all-optical neural networks via quantum interference
• Photonic systolic array for all-optical matrix-matrix multiplication
• Online training and pruning of multi-wavelength photonic neural networks
• Emergence of biased errors in imperfect photonic circuits