Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks

Abstract

Biological plastic neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifetime learning. The interplay of these elements leads to the emergence of adaptive behavior and intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed plastic neural networks with a large variety of dynamics, architectures, and plasticity rules: these artificial systems are composed of inputs, outputs, and plastic components that change in response to experiences in an environment. These systems may autonomously discover novel adaptive algorithms, and lead to hypotheses on the emergence of biological adaptation. EPANNs have seen considerable progress over the last two decades. Current scientific and technological advances in artificial neural networks are now setting the conditions for radically new approaches and results. In particular, the limitations of hand-designed networks could be overcome by more flexible and innovative solutions. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and developments are presented.

References in corpus (18)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning in Neural Networks: An Overview
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Overcoming catastrophic forgetting in neural networks
• LSTM: A Search Space Odyssey
• Theoretical Models of Learning to Learn
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• CMA-ES for Hyperparameter Optimization of Deep Neural Networks
• Neural Programmer-Interpreters
• Hierarchical Representations for Efficient Architecture Search
• A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data
• Evolution of Plastic Learning in Spiking Networks via Memristive Connections
• The Evolution of Learning Algorithms for Artificial Neural Networks
• Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks
• Evolving Spiking Networks with Variable Resistive Memories
• Convolution by Evolution: Differentiable Pattern Producing Networks
• Large Scale Evolution of Convolutional Neural Networks Using Volunteer Computing
• Computational models of long term plasticity and memory

Cited by in corpus (26)

• Learning to Continually Learn
• Evolving the pulmonary nodules diagnosis from classical approaches to deep learning aided decision support: three decades development course and future prospect
• Memorize or generalize? Searching for a compositional RNN in a haystack
• AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
• An Investigation of Replay-based Approaches for Continual Learning
• Meta-Learning through Hebbian Plasticity in Random Networks
• Evolving and Merging Hebbian Learning Rules: Increasing Generalization by Decreasing the Number of Rules
• Cutting-edge 3D Medical Image Segmentation Methods in 2020: Are Happy Families All Alike?
• Evolving the Behavior of Machines: From Micro to Macroevolution
• Exploring the Intersection between Neural Architecture Search and Continual Learning
• Learning from the Past: Continual Meta-Learning via Bayesian Graph Modeling
• Evolving Inborn Knowledge For Fast Adaptation in Dynamic POMDP Problems
• Meta-learnt priors slow down catastrophic forgetting in neural networks
• Finding online neural update rules by learning to remember
• Learning with Delayed Synaptic Plasticity
• Evolving Self-taught Neural Networks: The Baldwin Effect and the Emergence of Intelligence
• Evolving to learn: discovering interpretable plasticity rules for spiking networks
• Continual Learning with Fully Probabilistic Models
• Evolving Self-supervised Neural Networks: Autonomous Intelligence from Evolved Self-teaching
• Novelty Producing Synaptic Plasticity
• Evolutionary Computation and AI Safety: Research Problems Impeding Routine and Safe Real-world Application of Evolution
• Meta-Learning Bidirectional Update Rules
• Learning to Act through Evolution of Neural Diversity in Random Neural Networks
• Testing the Genomic Bottleneck Hypothesis in Hebbian Meta-Learning
• Minimal Neural Network Models for Permutation Invariant Agents
• Self-building Neural Networks