Improving Generalization in Meta Reinforcement Learning using Learned Objectives

Abstract

Biological evolution has distilled the experiences of many learners into the general learning algorithms of humans. Our novel meta reinforcement learning algorithm MetaGenRL is inspired by this process. MetaGenRL distills the experiences of many complex agents to meta-learn a low-complexity neural objective function that decides how future individuals will learn. Unlike recent meta-RL algorithms, MetaGenRL can generalize to new environments that are entirely different from those used for meta-training. In some cases, it even outperforms human-engineered RL algorithms. MetaGenRL uses off-policy second-order gradients during meta-training that greatly increase its sample efficiency.

Accepted to ICLR 2020

References in corpus (4)

• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Learning to reinforcement learn
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• Learning to Learn: Meta-Critic Networks for Sample Efficient Learning

Cited by in corpus (4)

• GeneraLight: Improving Environment Generalization of Traffic Signal Control via Meta Reinforcement Learning
• BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)
• A Brief Look at Generalization in Visual Meta-Reinforcement Learning
• Group Equivariant Deep Reinforcement Learning