Reprogrammable Electro-Optic Nonlinear Activation Functions for Optical Neural Networks

Abstract

We introduce an electro-optic hardware platform for nonlinear activation functions in optical neural networks. The optical-to-optical nonlinearity operates by converting a small portion of the input optical signal into an analog electric signal, which is used to intensity-modulate the original optical signal with no reduction in processing speed. Our scheme allows for complete nonlinear on-off contrast in transmission at relatively low optical power thresholds and eliminates the requirement of having additional optical sources between each layer of the network. Moreover, the activation function is reconfigurable via electrical bias, allowing it to be programmed or trained to synthesize a variety of nonlinear responses. Using numerical simulations, we demonstrate that this activation function significantly improves the expressiveness of optical neural networks, allowing them to perform well on two benchmark machine learning tasks: learning a multi-input exclusive-OR (XOR) logic function and classification of images of handwritten numbers from the MNIST dataset. The addition of the nonlinear activation function improves test accuracy on the MNIST task from 85% to 94%.

12 pages, 6 figures

References in corpus (10)

• Deep Learning with Coherent Nanophotonic Circuits
• Neuromorphic Silicon Photonic Networks
• Training of photonic neural networks through in situ backpropagation
• Self-Configuring Universal Linear Optical Component
• A silicon photonic modulator neuron
• Superconducting optoelectronic circuits for neuromorphic computing
• Matrix optimization on universal unitary photonic devices
• An Optical Frontend for a Convolutional Neural Network
• Principles of Neuromorphic Photonics
• Cavity-Enabled Self-Electro-Optic Bistability in Silicon Photonics

Cited by in corpus (40)

• Photonics for artificial intelligence and neuromorphic computing
• The physics of optical computing
• Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks
• Neuromorphic computing using wavelength-division multiplexing
• Single chip photonic deep neural network with accelerated training
• Prospects and applications of photonic neural networks
• Hardware error correction for programmable photonics
• Heuristic Recurrent Algorithms for Photonic Ising Machines
• Photonics for Neuromorphic Computing: Fundamentals, Devices, and Opportunities
• Arbitrary linear transformations for photons in the frequency synthetic dimension
• Asymptotically Fault-Tolerant Programmable Photonics
• All-Optical Spiking Neuron Based On Passive Micro-Resonator
• Order-of-magnitude differences in computational performance of analog Ising machines induced by the choice of nonlinearity
• Accurate Self-Configuration of Rectangular Multiport Interferometers
• Quantifying power use in silicon photonic neural networks
• Photonic convolutional neural networks using integrated diffractive optics
• Addressing the programming challenges of practical interferometric mesh based optical processors
• Scalable and Robust Photonic Integrated Unitary Converter
• Universal programmable photonic architecture for quantum information processing
• All-optical nonlinear activation function based on stimulated Brillouin scattering
• Optical Neural Network Based on Synthetic Nonlinear Photonic Lattices
• All-Optical Image Identification with Programmable Matrix Transformation
• On-chip Optical Phase Monitoring in Multi-Transverse-Mode Integrated Silicon-based Optical Processors
• Bayesian Photonic Accelerators for Energy Efficient and Noise Robust Neural Processing
• Optical neural network architecture for deep learning with the temporal synthetic dimension
• Metasurface-controlled holographic microcavities
• Using continuation methods to analyse the difficulty of problems solved by Ising machines
• Optical Neural Ordinary Differential Equations
• A Reconfigurable Linear RF Analog Processor for Realizing Microwave Artificial Neural Network
• Application of Reconfigurable All-Optical Activation Unit based on Optical Injection into Bistable Fabry-Pérot Laser in Multilayer Perceptron Neural Networks
• Parallel fault-tolerant programming of an arbitrary feedforward photonic network
• Channel Tiling for Improved Performance and Accuracy of Optical Neural Network Accelerators
• Image amplification in a self-imaging degenerate optical cavity
• The Impact of Feature Representation on the Accuracy of Photonic Neural Networks
• Unitary Learning for Deep Diffractive Neural Network
• Slimmed optical neural networks with multiplexed neuron sets and a corresponding backpropagation training algorithm
• Implementation of Optical Deep Neural Networks using the Fabry-Perot Interferometer
• A Comprehensive Survey on Nanophotonics Neural Networks
• Emergence of biased errors in imperfect photonic circuits
• All-Optical Doubly Resonant Cavities for ReLU Function in Nanophotonic Deep Learning