Coupled nonlinear delay systems as deep convolutional neural networks

Abstract

Neural networks are currently transforming the field of computer algorithms, yet their emulation on current computing substrates is highly inefficient. Reservoir computing was successfully implemented on a large variety of substrates and gave new insight in overcoming this implementation bottleneck. Despite its success, the approach lags behind the state of the art in deep learning. We therefore extend time-delay reservoirs to deep networks and demonstrate that these conceptually correspond to deep convolutional neural networks. Convolution is intrinsically realized on a substrate level by generic drive-response properties of dynamical systems. The resulting novelty is avoiding vector-matrix products between layers, which cause low efficiency in today's substrates. Compared to singleton time-delay reservoirs, our deep network achieves accuracy improvements by at least an order of magnitude in Mackey-Glass and Lorenz timeseries prediction.

References in corpus (10)

• Deep Learning with Coherent Nanophotonic Circuits
• All-Optical Machine Learning Using Diffractive Deep Neural Networks
• Recent Advances in Physical Reservoir Computing: A Review
• Optoelectronic Reservoir Computing
• Reinforcement Learning in a large scale photonic Recurrent Neural Network
• Attractor Reconstruction by Machine Learning
• Tutorial: Photonic Neural Networks in Delay Systems
• Embodiment of Learning in Electro-Optical Signal Processors
• The difference between memory and prediction in linear recurrent networks
• A new spatio-temporal description of long-delayed systems: ruling the dynamics

Cited by in corpus (6)

• Large-Scale Optical Reservoir Computing for Spatiotemporal Chaotic Systems Prediction
• Misalignment Resilient Diffractive Optical Networks
• Delay-induced self-oscillation excitation in the FitzHugh-Nagumo model: regular and chaotic dynamics
• Limit Cycles and Chaos Induced by a Nonlinearity with Memory
• Optical neuromorphic computing via temporal up-sampling and trainable encoding on a telecom device platform
• Deep Delay Loop Reservoir Computing for Specific Emitter Identification