Neural Episodic Control

Abstract

Deep reinforcement learning methods attain super-human performance in a wide range of environments. Such methods are grossly inefficient, often taking orders of magnitudes more data than humans to achieve reasonable performance. We propose Neural Episodic Control: a deep reinforcement learning agent that is able to rapidly assimilate new experiences and act upon them. Our agent uses a semi-tabular representation of the value function: a buffer of past experience containing slowly changing state representations and rapidly updated estimates of the value function. We show across a wide range of environments that our agent learns significantly faster than other state-of-the-art, general purpose deep reinforcement learning agents.

References in corpus (12)

• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Prioritized Experience Replay
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Deep Exploration via Bootstrapped DQN
• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Learning to Remember Rare Events
• Hybrid Reward Architecture for Reinforcement Learning
• Model-Free Episodic Control
• Control of Memory, Active Perception, and Action in Minecraft
• Safe and Efficient Off-Policy Reinforcement Learning
• Strategic Attentive Writer for Learning Macro-Actions
• Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening

Cited by in corpus (43)

• An Introduction to Variational Autoencoders
• Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
• Stabilizing Transformers for Reinforcement Learning
• ViZDoom Competitions: Playing Doom from Pixels
• Composable Planning with Attributes
• Replay in Deep Learning: Current Approaches and Missing Biological Elements
• Graph Element Networks: adaptive, structured computation and memory
• Analysing Results from AI Benchmarks: Key Indicators and How to Obtain Them
• Generalizable Episodic Memory for Deep Reinforcement Learning
• Learning to Learn Variational Semantic Memory
• Pointer Graph Networks
• Training Agents using Upside-Down Reinforcement Learning
• Concurrent Meta Reinforcement Learning
• A Short Survey On Memory Based Reinforcement Learning
• On the Estimation Bias in Double Q-Learning
• Memory-Based Graph Networks
• Offline Reinforcement Learning with Value-based Episodic Memory
• The Benchmark Lottery
• Deep Graph Memory Networks for Forgetting-Robust Knowledge Tracing
• DeepAveragers: Offline Reinforcement Learning by Solving Derived Non-Parametric MDPs
• Learning Guidance Rewards with Trajectory-space Smoothing
• Persistent Message Passing
• Model-Based Episodic Memory Induces Dynamic Hybrid Controls
• Continual and Multi-task Reinforcement Learning With Shared Episodic Memory
• Random Projection in Neural Episodic Control
• Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control in Computationally Complex Environments
• Biologically inspired architectures for sample-efficient deep reinforcement learning
• Memory-Efficient Episodic Control Reinforcement Learning with Dynamic Online k-means
• Sample-Efficient Reinforcement Learning with Maximum Entropy Mellowmax Episodic Control
• Learning Associative Inference Using Fast Weight Memory
• A Memory-Augmented Neural Network Model of Abstract Rule Learning
• Self-Imitation Learning for Robot Tasks with Sparse and Delayed Rewards
• Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memory
• A Proposal for Intelligent Agents with Episodic Memory
• Reinforcement Learning With Sparse-Executing Actions via Sparsity Regularization
• Successor Feature Neural Episodic Control
• Theoretically Principled Deep RL Acceleration via Nearest Neighbor Function Approximation
• State-based Episodic Memory for Multi-Agent Reinforcement Learning
• Multitask Adaptation by Retrospective Exploration with Learned World Models
• CNN with large memory layers
• Improving Sample Efficiency with Normalized RBF Kernels
• BOOK: Storing Algorithm-Invariant Episodes for Deep Reinforcement Learning
• Rapid Task-Solving in Novel Environments