Population-Based Evolution Optimizes a Meta-Learning Objective

Abstract

Meta-learning models, or models that learn to learn, have been a long-desired target for their ability to quickly solve new tasks. Traditional meta-learning methods can require expensive inner and outer loops, thus there is demand for algorithms that discover strong learners without explicitly searching for them. We draw parallels to the study of evolvable genomes in evolutionary systems -- genomes with a strong capacity to adapt -- and propose that meta-learning and adaptive evolvability optimize for the same objective: high performance after a set of learning iterations. We argue that population-based evolutionary systems with non-static fitness landscapes naturally bias towards high-evolvability genomes, and therefore optimize for populations with strong learning ability. We demonstrate this claim with a simple evolutionary algorithm, Population-Based Meta Learning (PBML), that consistently discovers genomes which display higher rates of improvement over generations, and can rapidly adapt to solve sparse fitness and robotic control tasks.

References in corpus (18)

• Theoretical Models of Learning to Learn
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Meta-Learning for Semi-Supervised Few-Shot Classification
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Learning to Learn without Gradient Descent by Gradient Descent
• Meta-Reinforcement Learning of Structured Exploration Strategies
• Evolved Policy Gradients
• Meta Learning Shared Hierarchies
• Online Meta-Learning
• How to train your MAML
• Online Learning Rate Adaptation with Hypergradient Descent
• Evolvability Is Inevitable: Increasing Evolvability Without the Pressure to Adapt
• ES-MAML: Simple Hessian-Free Meta Learning
• Understanding Short-Horizon Bias in Stochastic Meta-Optimization
• Meta-Learning by the Baldwin Effect
• Probabilistic Model-Agnostic Meta-Learning
• Learning To Simulate