Single-chip photonic deep neural network for instantaneous image classification

Abstract

Deep neural networks with applications from computer vision and image processing to medical diagnosis are commonly implemented using clock-based processors, where computation speed is limited by the clock frequency and the memory access time. Advances in photonic integrated circuits have enabled research in photonic computation, where, despite excellent features such as fast linear computation, no integrated photonic deep network has been demonstrated to date due to the lack of scalable nonlinear functionality and the loss of photonic devices, making scalability to a large number of layers challenging. Here we report the first integrated end-to-end photonic deep neural network (PDNN) that performs instantaneous image classification through direct processing of optical waves. Images are formed on the input pixels and optical waves are coupled into nanophotonic waveguides and processed as the light propagates through layers of neurons on-chip. Each neuron generates an optical output from input optical signals, where linear computation is performed optically and the nonlinear activation function is realised opto-electronically. The output of a laser coupled into the chip is uniformly distributed among all neurons within the network providing the same per-neuron supply light. Thus, all neurons have the same optical output range enabling scalability to deep networks with large number of layers. The PDNN chip is used for 2- and 4-class classification of handwritten letters achieving accuracies of higher than 93.7% and 90.3%, respectively, with a computation time less than one clock cycle of state-of-the-art digital computation platforms. Direct clock-less processing of optical data eliminates photo-detection, A/D conversion, and the requirement for a large memory module, enabling significantly faster and more energy-efficient neural networks for the next generations of deep learning systems.

References in corpus (3)

• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Photonics for artificial intelligence and neuromorphic computing
• 11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks

Cited by in corpus (53)

• The physics of optical computing
• Applications of integrated optical microcombs
• Intelligent Computing: The Latest Advances, Challenges and Future
• Image sensing with multilayer, nonlinear optical neural networks
• Single chip photonic deep neural network with accelerated training
• Neuromorphic computing using wavelength-division multiplexing
• Deep Learning with Coherent VCSEL Neural Networks
• High-order tensor flow processing using integrated photonic circuits
• Sagnac interference in integrated photonics for reflection mirrors, gyroscopes, filters, and wavelength interleavers
• In-memory photonic dot-product engine with electrically programmable weight banks
• Femtojoule, femtosecond all-optical switching in lithium niobate nanophotonics
• Photonics for Neuromorphic Computing: Fundamentals, Devices, and Opportunities
• Why optics needs thickness
• Silicon Photonic Architecture for Training Deep Neural Networks with Direct Feedback Alignment
• Massively Parallel Universal Linear Transformations using a Wavelength-Multiplexed Diffractive Optical Network
• Integrated Photonic Tensor Processing Unit for a Matrix Multiply: a Review
• Photonic neural networks based on integrated silicon microresonators
• Photonic real time video image signal processor at 17Tb/s based on a Kerr microcomb
• Optical Generative Models
• Deep Photonic Reservoir Computer Based on Frequency Multiplexing with Fully Analog Connection Between Layers
• An array of microresonators as a Photonic Extreme Learning Machine
• Programmable wave-based analog computing machine: a metastructure that designs metastructures
• Quantum-limited stochastic optical neural networks operating at a few quanta per activation
• Ultrafast single-channel machine vision based on neuro-inspired photonic computing
• The Goldilocks Principle of Learning Unitaries by Interlacing Fixed Operators with Programmable Phase Shifters on a Photonic Chip
• Deep Photonic Networks with Arbitrary and Broadband Functionality
• Multiplexed all-optical permutation operations using a reconfigurable diffractive optical network
• An optoacoustic field-programmable perceptron for recurrent neural networks
• Bound-state-in-continuum guided modes in a multilayer electro-optically active photonic integrated circuit platform
• Waveguide-multiplexed photonic matrix-vector multiplication processor using multiport photodetectors
• Asymmetrical estimator for training encapsulated deep photonic neural networks
• Unitary control of partially coherent waves. II. Transmission or reflection
• 16-channel Photonic Solver for Optimization Problems on a Silicon Chip
• AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks
• Neural Network Methods for Radiation Detectors and Imaging
• Physical Neural Networks with Self-Learning Capabilities
• Inverse Design of Unitary Transmission Matrices in Silicon Photonic Coupled Waveguide Arrays using a Neural Adjoint Model
• Single-Shot Matrix-Matrix Multiplication Optical Tensor Processor for Deep Learning
• High-speed coherent photonic random-access memory in long-lasting sound waves
• A Reconfigurable Linear RF Analog Processor for Realizing Microwave Artificial Neural Network
• Broadband Unidirectional Visible Imaging Using Wafer-Scale Nano-Fabrication of Multi-Layer Diffractive Optical Processors
• Arbitrary control over multimode wave propagation for machine learning
• Non-volatile hybrid optical phase shifter driven by a ferroelectric transistor
• A 262 TOPS Hyperdimensional Photonic AI Accelerator powered by a Si3N4 microcomb laser
• Photonic systolic array for all-optical matrix-matrix multiplication
• The Impact of Feature Representation on the Accuracy of Photonic Neural Networks
• Passive All-Optical Nonlinear Neuron Activation via PPLN Nanophotonic Waveguides
• Online training and pruning of multi-wavelength photonic neural networks
• Quantum-secure multiparty deep learning
• Neural networks with quantum states of light
• On-chip arbitrary dispersion engineering with deep photonic networks
• Perspective of high-speed Mach-Zehnder modulators based on nonlinear optics and complex band structures
• Forming complex neurons by four-wave mixing in a Bose-Einstein condensate