An Electro-Photonic System for Accelerating Deep Neural Networks

Abstract

The number of parameters in deep neural networks (DNNs) is scaling at about 5$\times$ the rate of Moore's Law. To sustain this growth, photonic computing is a promising avenue, as it enables higher throughput in dominant general matrix-matrix multiplication (GEMM) operations in DNNs than their electrical counterpart. However, purely photonic systems face several challenges including lack of photonic memory and accumulation of noise. In this paper, we present an electro-photonic accelerator, ADEPT, which leverages a photonic computing unit for performing GEMM operations, a vectorized digital electronic ASIC for performing non-GEMM operations, and SRAM arrays for storing DNN parameters and activations. In contrast to prior works in photonic DNN accelerators, we adopt a system-level perspective and show that the gains while large are tempered relative to prior expectations. Our goal is to encourage architects to explore photonic technology in a more pragmatic way considering the system as a whole to understand its general applicability in accelerating today's DNNs. Our evaluation shows that ADEPT can provide, on average, 5.73$\times$ higher throughput per Watt compared to the traditional systolic arrays (SAs) in a full-system, and at least 6.8$\times$ and $2.5\times$ better throughput per Watt, compared to state-of-the-art electronic and photonic accelerators, respectively.

References in corpus (31)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Deep Learning with Coherent Nanophotonic Circuits
• Language Models are Few-Shot Learners
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Parallel convolution processing using an integrated photonic tensor core
• Photonics for artificial intelligence and neuromorphic computing
• 11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks
• cuDNN: Efficient Primitives for Deep Learning
• Neuromorphic Silicon Photonic Networks
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit
• Self-Configuring Universal Linear Optical Component
• All Optical Neural Network with Nonlinear Activation Functions
• Efficient, Compact and Low Loss Thermo-Optic Phase Shifter in Silicon
• A silicon photonic modulator neuron
• Electric Field-Induced Second Order Nonlinear Optical Effects in Silicon Waveguides
• Digital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs)
• Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation
• Hardware error correction for programmable photonics
• Origami: A 803 GOp/s/W Convolutional Network Accelerator
• 60dB high-extinction auto-configured Mach--Zehnder interferometer
• Scaling Up Silicon Photonic-based Accelerators: Challenges and Opportunities
• Broadband nanoelectromechanical phase shifting of light on a chip
• Photoconductive Heaters Enable Control of Large-Scale Silicon Photonic Ring Resonator Circuits
• PCNNA: A Photonic Convolutional Neural Network Accelerator
• Accurate Self-Configuration of Rectangular Multiport Interferometers
• Low-memory GEMM-based convolution algorithms for deep neural networks
• Stability of Self-Configuring Large Multiport Interferometers
• Streaming End-to-end Speech Recognition For Mobile Devices
• Optical Convolutional Neural Networks -- Combining Silicon Photonics and Fourier Optics for Computer Vision

Cited by in corpus (4)

• Mirage: An RNS-Based Photonic Accelerator for DNN Training
• A Blueprint for Precise and Fault-Tolerant Analog Neural Networks
• AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks
• Architecture-Level Modeling of Photonic Deep Neural Network Accelerators