Meta Learning Black-Box Population-Based Optimizers
arXiv:2103.03526
Abstract
The no free lunch theorem states that no model is better suited to every problem. A question that arises from this is how to design methods that propose optimizers tailored to specific problems achieving state-of-the-art performance. This paper addresses this issue by proposing the use of meta-learning to infer population-based black-box optimizers that can automatically adapt to specific classes of problems. We suggest a general modeling of population-based algorithms that result in Learning-to-Optimize POMDP (LTO-POMDP), a meta-learning framework based on a specific partially observable Markov decision process (POMDP). From that framework's formulation, we propose to parameterize the algorithm using deep recurrent neural networks and use a meta-loss function based on stochastic algorithms' performance to train efficient data-driven optimizers over several related optimization tasks. The learned optimizers' performance based on this implementation is assessed on various black-box optimization tasks and hyperparameter tuning of machine learning models. Our results revealed that the meta-loss function encourages a learned algorithm to alter its search behavior so that it can easily fit into a new context. Thus, it allows better generalization and higher sample efficiency than state-of-the-art generic optimization algorithms, such as the Covariance matrix adaptation evolution strategy (CMA-ES).
9 pages, 7 figures
References in corpus (15)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Prototypical Networks for Few-shot Learning
- Weight Uncertainty in Neural Networks
- Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
- On First-Order Meta-Learning Algorithms
- RL: Fast Reinforcement Learning via Slow Reinforcement Learning
- Meta-Learning with Implicit Gradients
- Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles
- Learning to Learn without Gradient Descent by Gradient Descent
- Meta reinforcement learning as task inference
- IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics
- Exploring the Evolution of GANs through Quality Diversity
- Using a thousand optimization tasks to learn hyperparameter search strategies
- Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves