Multiplex visibility graphs to investigate recurrent neural networks dynamics

Abstract

A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning of such hyperparameters may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize the internal RNN dynamics. Through this insight, we are able to design a principled unsupervised method to derive configurations with maximized performances, in terms of prediction error and memory capacity. In particular, we propose to model time series of neurons activations with the recently introduced horizontal visibility graphs, whose topological properties reflect important dynamical features of the underlying dynamic system. Successively, each graph becomes a layer of a larger structure, called multiplex. We show that topological properties of such a multiplex reflect important features of RNN dynamics and are used to guide the tuning procedure. To validate the proposed method, we consider a class of RNNs called echo state networks. We perform experiments and discuss results on several benchmarks and real-world dataset of call data records.

References in corpus (24)

• Deep Learning in Neural Networks: An Overview
• Assortative mixing in networks
• Multilayer Networks
• The structure and dynamics of multilayer networks
• Characterization of complex networks: A survey of measurements
• Mathematical Formulation of Multi-Layer Networks
• Are biological systems poised at criticality?
• Generalizations of the clustering coefficient to weighted complex networks
• Griffiths phases and the stretching of criticality in brain networks
• Stimulus-Dependent Suppression of Chaos in Recurrent Neural Networks
• Power laws and Self-Organized Criticality in Theory and Nature
• Description of stochastic and chaotic series using visibility graphs
• Network structure of multivariate time series
• Weighted Multiplex Networks
• Data for Development: the D4D Challenge on Mobile Phone Data
• Approximating the largest eigenvalue of network adjacency matrices
• Feigenbaum graphs: a complex network perspective of chaos
• Investigating echo state networks dynamics by means of recurrence analysis
• Distinguishing noise from chaos: objective versus subjective criteria using Horizontal Visibility Graph
• Horizontal visibility graphs transformed from fractional Brownian motions: Topological properties versus Hurst index
• Analytical properties of horizontal visibility graphs in the Feigenbaum scenario
• Determination of the edge of criticality in echo state networks through Fisher information maximization
• Revisiting chaos in stimulus-driven spiking networks: signal encoding and discrimination
• Quasiperiodic graphs at the onset of chaos

Cited by in corpus (10)

• Complex network approaches to nonlinear time series analysis
• Time Series Analysis via Network Science: Concepts and Algorithms
• Visibility graph analysis of economy policy uncertainty indices
• Mapping images into ordinal networks
• Multiplex Visibility Graphs as a complementary tool for describing the relation between ground level O3 and NO2
• Tetradic motif profiles of horizontal visibility graphs
• Analytic degree distributions of horizontal visibility graphs mapped from unrelated random series and multifractal binomial measures
• Mapping Coupled Time-series Onto Complex Network
• Triadic time series motifs
• Visibility graph analysis of the grains and oilseeds indices