ES-MAML: Simple Hessian-Free Meta Learning

Abstract

We introduce ES-MAML, a new framework for solving the model agnostic meta learning (MAML) problem based on Evolution Strategies (ES). Existing algorithms for MAML are based on policy gradients, and incur significant difficulties when attempting to estimate second derivatives using backpropagation on stochastic policies. We show how ES can be applied to MAML to obtain an algorithm which avoids the problem of estimating second derivatives, and is also conceptually simple and easy to implement. Moreover, ES-MAML can handle new types of nonsmooth adaptation operators, and other techniques for improving performance and estimation of ES methods become applicable. We show empirically that ES-MAML is competitive with existing methods and often yields better adaptation with fewer queries.

arXiv commentPublished as a conference paper in ICLR 2020. Code can be found in http://github.com/google-research/google-research/tree/master/es_maml

References in corpus (8)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
• Evolutionary Algorithms for Reinforcement Learning
• Simple random search provides a competitive approach to reinforcement learning
• Meta-Learning by the Baldwin Effect
• Unbiased Simulation for Optimizing Stochastic Function Compositions
• Structured Monte Carlo Sampling for Nonisotropic Distributions via Determinantal Point Processes

Cited by in corpus (14)

• Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization
• Inexact-ADMM Based Federated Meta-Learning for Fast and Continual Edge Learning
• Task-Robust Model-Agnostic Meta-Learning
• On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning
• Few-shot Quality-Diversity Optimization
• Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD
• Adaptive Asynchronous Control Using Meta-learned Neural Ordinary Differential Equations
• Distributed Evolution Strategies Using TPUs for Meta-Learning
• Population-Based Evolution Optimizes a Meta-Learning Objective
• Testing the Genomic Bottleneck Hypothesis in Hebbian Meta-Learning
• Neural Auto-Curricula
• Meta Learning in the Continuous Time Limit
• Learning Synthetic to Real Transfer for Localization and Navigational Tasks
• Compositional federated learning: Applications in distributionally robust averaging and meta learning