Optimal modularity and memory capacity of neural reservoirs

Abstract

The neural network is a powerful computing framework that has been exploited by biological evolution and by humans for solving diverse problems. Although the computational capabilities of neural networks are determined by their structure, the current understanding of the relationships between a neural network's architecture and function is still primitive. Here we reveal that neural network's modular architecture plays a vital role in determining the neural dynamics and memory performance of the network of threshold neurons. In particular, we demonstrate that there exists an optimal modularity for memory performance, where a balance between local cohesion and global connectivity is established, allowing optimally modular networks to remember longer. Our results suggest that insights from dynamical analysis of neural networks and information spreading processes can be leveraged to better design neural networks and may shed light on the brain's modular organization.

References in corpus (7)

• Deep Learning in Neural Networks: An Overview
• Benchmark graphs for testing community detection algorithms
• Structure and tie strengths in mobile communication networks
• Kronecker Graphs: An Approach to Modeling Networks
• Spatial Growth of Real-world Networks
• Optimal hierarchical modular topologies for producing limited sustained activation of neural networks
• Optimal modularity and memory capacity of neural reservoirs

Cited by in corpus (9)

• Biological neurons act as generalization filters in reservoir computing
• Optimal modularity and memory capacity of neural reservoirs
• Exploiting Multiple Timescales in Hierarchical Echo State Networks
• Periodic structure of memory function in spintronics reservoir with feedback current
• Antivax movement and epidemic spreading in the era of social networks: Nonmonotonic effects, bistability and network segregation
• The information content of brain states is explained by structural constraints on state energetics
• Optimal interplay between synaptic strengths and network structure enhances activity fluctuations and information propagation in hierarchical modular networks
• Information Evolution in Complex Networks
• Prediction performance of random reservoirs with different topology for nonlinear dynamical systems with different number of degrees of freedom