Boolean learning under noise-perturbations in hardware neural networks
arXiv:2003.12319 · doi:10.1515/nanoph-2020-0171
Abstract
A high efficiency hardware integration of neural networks benefits from realizing nonlinearity, network connectivity and learning fully in a physical substrate. Multiple systems have recently implemented some or all of these operations, yet the focus was placed on addressing technological challenges. Fundamental questions regarding learning in hardware neural networks remain largely unexplored. Noise in particular is unavoidable in such architectures, and here we investigate its interaction with a learning algorithm using an opto-electronic recurrent neural network. We find that noise strongly modifies the system's path during convergence, and surprisingly fully decorrelates the final readout weight matrices. This highlights the importance of understanding architecture, noise and learning algorithm as interacting players, and therefore identifies the need for mathematical tools for noisy, analogue system optimization.
arXiv comment8 pages, 5 figures
References in corpus (12)
- Deep Learning with Coherent Nanophotonic Circuits
- All-Optical Machine Learning Using Diffractive Deep Neural Networks
- All-optical Reservoir Computing
- Reinforcement Learning in a large scale photonic Recurrent Neural Network
- Surprises in High-Dimensional Ridgeless Least Squares Interpolation
- Reply to "Comment on `Inference with minimal Gibbs free energy in information field theory'" by Iatsenko, Stefanovska and McClintock
- Embodiment of Learning in Electro-Optical Signal Processors
- Fundamental aspects of noise in analog-hardware neural networks
- Training Passive Photonic Reservoirs with Integrated Optical Readout
- Diffractive coupling for photonic networks: how big can we go?
- Competitive Brownian and Levy walkers
- Computing optimal discrete readout weights in reservoir computing is NP-hard