Livewired Neural Networks: Making Neurons That Fire Together Wire Together

Abstract

Until recently, artificial neural networks were typically designed with a fixed network structure. Here, I argue that network structure is highly relevant to function, and therefore neural networks should be livewired (Eagleman 2020): dynamically rewired to reflect relationships between higher order representations of the external environment identified by coincident activations in individual neurons. I discuss how this approach may enable such networks to build compositional world models that operate on symbols and that achieve few-shot learning, capabilities thought by many to be critical to human-level cognition. Here, I also 1) discuss how such livewired neural networks maximize the information the environment provides to a model, 2) explore evidence indicating that livewiring is implemented in the brain, guided by glial cells, 3) discuss how livewiring may give rise to the associative emergent behaviors of brains, and 4) suggest paths for future research using livewired networks to understand and create human-like reasoning.

34 pages, 1 figure

References in corpus (10)

• Improving neural networks by preventing co-adaptation of feature detectors
• Going Deeper with Convolutions
• To prune, or not to prune: exploring the efficacy of pruning for model compression
• Graph Transformer Networks
• Practical recommendations for gradient-based training of deep architectures
• The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision
• Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization
• On the Binding Problem in Artificial Neural Networks
• Neural Relational Inference with Fast Modular Meta-learning
• Visual Concepts and Compositional Voting