Been There, Done That: Meta-Learning with Episodic Recall

Abstract

Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks reoccur - as they do in natural environments - metalearning agents must explore again instead of immediately exploiting previously discovered solutions. We propose a formalism for generating open-ended yet repetitious environments, then develop a meta-learning architecture for solving these environments. This architecture melds the standard LSTM working memory with a differentiable neural episodic memory. We explore the capabilities of agents with this episodic LSTM in five meta-learning environments with reoccurring tasks, ranging from bandits to navigation and stochastic sequential decision problems.

arXiv commentICML 2018

References in corpus (7)

• Overcoming catastrophic forgetting in neural networks
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Matching Networks for One Shot Learning
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Learning to reinforcement learn
• Learning to Remember Rare Events
• Model-Free Episodic Control

Cited by in corpus (10)

• Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
• Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL
• Concurrent Meta Reinforcement Learning
• Synthetic Returns for Long-Term Credit Assignment
• MetaPred: Meta-Learning for Clinical Risk Prediction with Limited Patient Electronic Health Records
• Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrifices
• Distributed Multi-agent Meta Learning for Trajectory Design in Wireless Drone Networks
• Robust Domain Randomised Reinforcement Learning through Peer-to-Peer Distillation
• The Emergence of Abstract and Episodic Neurons in Episodic Meta-RL
• Rapid Task-Solving in Novel Environments