Different eigenvalue distributions encode the same temporal tasks in recurrent neural networks
arXiv:2005.13074 · doi:10.1007/s11571-022-09802-5
Abstract
Different brain areas, such as the cortex and, more specifically, the prefrontal cortex, show great recurrence in their connections, even in early sensory areas. {Several approaches and methods based on trained networks have been proposed to model and describe these regions. It is essential to understand the dynamics behind the models because they are used to build different hypotheses about the functioning of brain areas and to explain experimental results. The main contribution here is the description of the dynamics through the classification and interpretation carried out with a set of numerical simulations. This study sheds light on the multiplicity of solutions obtained for the same tasks and shows the link between the spectra of linearized trained networks and the dynamics of the counterparts. The patterns in the distribution of the eigenvalues of the recurrent weight matrix were studied and properly related to the dynamics in each task.
19 pages, 10 Figures
References in corpus (10)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- full-FORCE: A Target-Based Method for Training Recurrent Networks
- Correlations between synapses in pairs of neurons slow down dynamics in randomly connected neural networks
- Understanding the computation of time using neural network models
- A State Space Approach for Piecewise-Linear Recurrent Neural Networks for Reconstructing Nonlinear Dynamics from Neural Measurements
- Training dynamically balanced excitatory-inhibitory networks
- Eigenvalue Repulsion and Eigenfunction Localization in Sparse Non-Hermitian Random Matrices
- From synaptic interactions to collective dynamics in random neuronal networks models: critical role of eigenvectors and transient behavior
- How Robust are Deep Neural Networks?
- Recurrent Neural Network Learning of Performance and Intrinsic Population Dynamics from Sparse Neural Data