Interpreting recurrent neural networks behaviour via excitable network attractors

Abstract

Introduction: Machine learning provides fundamental tools both for scientific research and for the development of technologies with significant impact on society. It provides methods that facilitate the discovery of regularities in data and that give predictions without explicit knowledge of the rules governing a system. However, a price is paid for exploiting such flexibility: machine learning methods are typically black-boxes where it is difficult to fully understand what the machine is doing or how it is operating. This poses constraints on the applicability and explainability of such methods. Methods: Our research aims to open the black-box of recurrent neural networks, an important family of neural networks used for processing sequential data. We propose a novel methodology that provides a mechanistic interpretation of behaviour when solving a computational task. Our methodology uses mathematical constructs called excitable network attractors, which are invariant sets in phase space composed of stable attractors and excitable connections between them. Results and Discussion: As the behaviour of recurrent neural networks depends both on training and on inputs to the system, we introduce an algorithm to extract network attractors directly from the trajectory of a neural network while solving tasks. Simulations conducted on a controlled benchmark task confirm the relevance of these attractors for interpreting the behaviour of recurrent neural networks, at least for tasks that involve learning a finite number of stable states and transitions between them.

arXiv commentrevised version

References in corpus (17)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• An overview of gradient descent optimization algorithms
• On the difficulty of training Recurrent Neural Networks
• Methods for Interpreting and Understanding Deep Neural Networks
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data
• Linking connectivity, dynamics and computations in low-rank recurrent neural networks
• Stimulus-Dependent Suppression of Chaos in Recurrent Neural Networks
• full-FORCE: A Target-Based Method for Training Recurrent Networks
• Investigating echo state networks dynamics by means of recurrence analysis
• Local Dynamics in Trained Recurrent Neural Networks
• Determination of the edge of criticality in echo state networks through Fisher information maximization
• On designing heteroclinic networks from graphs
• Training Echo State Networks with Regularization through Dimensionality Reduction
• On the low dimensional dynamics of structured random networks
• Designing heteroclinic and excitable networks in phase space using two populations of coupled cells
• Orthogonal Echo State Networks and stochastic evaluations of likelihoods

Cited by in corpus (8)

• Continual Learning for Recurrent Neural Networks: an Empirical Evaluation
• Chaos on compact manifolds: Differentiable synchronizations beyond the Takens theorem
• Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere
• The Echo Index and multistability in input-driven recurrent neural networks
• Patched patterns and emergence of chaotic interfaces in arrays of nonlocally coupled excitable systems
• On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysis
• Scope and Sense of Explainability for AI-Systems
• Reinforcement Learning with Intrinsic Affinity for Personalized Prosperity Management