Model-Based Episodic Memory Induces Dynamic Hybrid Controls

Abstract

Episodic control enables sample efficiency in reinforcement learning by recalling past experiences from an episodic memory. We propose a new model-based episodic memory of trajectories addressing current limitations of episodic control. Our memory estimates trajectory values, guiding the agent towards good policies. Built upon the memory, we construct a complementary learning model via a dynamic hybrid control unifying model-based, episodic and habitual learning into a single architecture. Experiments demonstrate that our model allows significantly faster and better learning than other strong reinforcement learning agents across a variety of environments including stochastic and non-Markovian settings.

arXiv comment26 pages

References in corpus (13)

• Prioritized Experience Replay
• Deep Recurrent Q-Learning for Partially Observable MDPs
• Recurrent World Models Facilitate Policy Evolution
• Learning Latent Dynamics for Planning from Pixels
• Model-Free Episodic Control
• Unsupervised Predictive Memory in a Goal-Directed Agent
• Neural Episodic Control
• Generalization of Reinforcement Learners with Working and Episodic Memory
• Generalizable Episodic Memory for Deep Reinforcement Learning
• Fast deep reinforcement learning using online adjustments from the past
• Learning to Remember More with Less Memorization
• Episodic Memory Deep Q-Networks
• Neurocoder: Learning General-Purpose Computation Using Stored Neural Programs