Evolving and Merging Hebbian Learning Rules: Increasing Generalization by Decreasing the Number of Rules
arXiv:2104.07959 · doi:10.1145/3449639.3459317
Abstract
Generalization to out-of-distribution (OOD) circumstances after training remains a challenge for artificial agents. To improve the robustness displayed by plastic Hebbian neural networks, we evolve a set of Hebbian learning rules, where multiple connections are assigned to a single rule. Inspired by the biological phenomenon of the genomic bottleneck, we show that by allowing multiple connections in the network to share the same local learning rule, it is possible to drastically reduce the number of trainable parameters, while obtaining a more robust agent. During evolution, by iteratively using simple K-Means clustering to combine rules, our Evolve and Merge approach is able to reduce the number of trainable parameters from 61,440 to 1,920, while at the same time improving robustness, all without increasing the number of generations used. While optimization of the agents is done on a standard quadruped robot morphology, we evaluate the agents' performances on slight morphology modifications in a total of 30 unseen morphologies. Our results add to the discussion on generalization, overfitting and OOD adaptation. To create agents that can adapt to a wider array of unexpected situations, Hebbian learning combined with a regularising "genomic bottleneck" could be a promising research direction.
References in corpus (18)
- Scikit-learn: Machine Learning in Python
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Robots that can adapt like animals
- A Study on Overfitting in Deep Reinforcement Learning
- Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks
- Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation
- Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask
- HyperNetworks
- A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning
- Scaling MAP-Elites to Deep Neuroevolution
- Neuroevolution of Self-Interpretable Agents
- Investigating Generalisation in Continuous Deep Reinforcement Learning
- Evolving Inborn Knowledge For Fast Adaptation in Dynamic POMDP Problems
- Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking
- Learning with Delayed Synaptic Plasticity
- Network of Evolvable Neural Units: Evolving to Learn at a Synaptic Level