AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks

Abstract

AnalogVNN, a simulation framework built on PyTorch which can simulate the effects of optoelectronic noise, limited precision, and signal normalization present in photonic neural network accelerators. We use this framework to train and optimize linear and convolutional neural networks with up to 9 layers and ~1.7 million parameters, while gaining insights into how normalization, activation function, reduced precision, and noise influence accuracy in analog photonic neural networks. By following the same layer structure design present in PyTorch, the AnalogVNN framework allows users to convert most digital neural network models to their analog counterparts with just a few lines of code, taking full advantage of the open-source optimization, deep learning, and GPU acceleration libraries available through PyTorch. Code is available at https://analogvnn.github.io

arXiv comment28 pages; replace figure 6C; better format; updated links

References in corpus (23)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Deep Learning with Coherent Nanophotonic Circuits
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Parallel convolution processing using an integrated photonic tensor core
• Photonics for artificial intelligence and neuromorphic computing
• 11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks
• Single-chip photonic deep neural network for instantaneous image classification
• Attojoule Optoelectronics for Low-Energy Information Processing and Communications: a Tutorial Review
• Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit
• Train longer, generalize better: closing the generalization gap in large batch training of neural networks
• Large-Scale Optical Neural Networks based on Photoelectric Multiplication
• Programmable Phase-change Metasurfaces on Waveguides for Multimode Photonic Convolutional Neural Network
• An optical neural network using less than 1 photon per multiplication
• How Does Batch Normalization Help Optimization?
• A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays
• Mixed-precision deep learning based on computational memory
• Random Projections through multiple optical scattering: Approximating kernels at the speed of light
• Scaling Up Silicon Photonic-based Accelerators: Challenges and Opportunities
• Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization
• An Electro-Photonic System for Accelerating Deep Neural Networks
• The Effects of Hyperparameters on SGD Training of Neural Networks
• Quantifying power use in silicon photonic neural networks
• Hyperparameter Tuning for Deep Reinforcement Learning Applications