Learn to Synchronize, Synchronize to Learn

Abstract

In recent years, the machine learning community has seen a continuous growing interest in research aimed at investigating dynamical aspects of both training procedures and machine learning models. Of particular interest among recurrent neural networks we have the Reservoir Computing (RC) paradigm characterized by conceptual simplicity and a fast training scheme. Yet, the guiding principles under which RC operates are only partially understood. In this work, we analyze the role played by Generalized Synchronization (GS) when training a RC to solve a generic task. In particular, we show how GS allows the reservoir to correctly encode the system generating the input signal into its dynamics. We also discuss necessary and sufficient conditions for the learning to be feasible in this approach. Moreover, we explore the role that ergodicity plays in this process, showing how its presence allows the learning outcome to apply to multiple input trajectories. Finally, we show that satisfaction of the GS can be measured by means of the Mutual False Nearest Neighbors index, which makes effective to practitioners theoretical derivations.

References in corpus (19)

• Recent Advances in Physical Reservoir Computing: A Review
• A Robust Method for Detecting Interdependences: Application to Intracranially Recorded EEG
• Attractor Reconstruction by Machine Learning
• Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM
• Embedding and Approximation Theorems for Echo State Networks
• Investigating echo state networks dynamics by means of recurrence analysis
• Local Dynamics in Trained Recurrent Neural Networks
• Training Echo State Networks with Regularization through Dimensionality Reduction
• Mean Field Theory of Dynamical Systems Driven by External Signals
• Consistency in Echo-State Networks
• Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective
• Chaos on compact manifolds: Differentiable synchronizations beyond the Takens theorem
• The difference between memory and prediction in linear recurrent networks
• Accuracy of neural networks for the simulation of chaotic dynamics: precision of training data vs precision of the algorithm
• Memory and forecasting capacities of nonlinear recurrent networks
• The Echo Index and multistability in input-driven recurrent neural networks
• Memory and Information Processing in Recurrent Neural Networks
• Dynamical Systems as Temporal Feature Spaces
• Forecasting Using Reservoir Computing: The Role of Generalized Synchronization

Cited by in corpus (5)

• Learning strange attractors with reservoir systems
• Predicting Critical Transitions in Multiscale Dynamical Systems Using Reservoir Computing
• Criticality in Reservoir Computer of Coupled Phase Oscillators
• Learning unidirectional coupling using echo-state network
• Input-dependence in quantum reservoir computing