Testing the Genomic Bottleneck Hypothesis in Hebbian Meta-Learning
arXiv:2011.06811
Abstract
Hebbian meta-learning has recently shown promise to solve hard reinforcement learning problems, allowing agents to adapt to some degree to changes in the environment. However, because each synapse in these approaches can learn a very specific learning rule, the ability to generalize to very different situations is likely reduced. We hypothesize that limiting the number of Hebbian learning rules through a "genomic bottleneck" can act as a regularizer leading to better generalization across changes to the environment. We test this hypothesis by decoupling the number of Hebbian learning rules from the number of synapses and systematically varying the number of Hebbian learning rules. The results in this paper suggest that simultaneously learning the Hebbian learning rules and their assignment to synapses is a difficult optimization problem, leading to poor performance in the environments tested. However, parallel research to ours finds that it is indeed possible to reduce the number of learning rules by clustering similar rules together. How to best implement a "genomic bottleneck" algorithm is thus an important research direction that warrants further investigation.
arXiv commentJMLR 148, NeurIPS pre-registration workshop 2020
References in corpus (16)
- Adam: A Method for Stochastic Optimization
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
- Dota 2 with Large Scale Deep Reinforcement Learning
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- A Simple Neural Attentive Meta-Learner
- Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
- On First-Order Meta-Learning Algorithms
- RL: Fast Reinforcement Learning via Slow Reinforcement Learning
- Learning to reinforcement learn
- Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks
- Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation
- ES-MAML: Simple Hessian-Free Meta Learning
- Metalearning with Hebbian Fast Weights
- Evolving and Merging Hebbian Learning Rules: Increasing Generalization by Decreasing the Number of Rules