Deep Attention Recurrent Q-Network

Abstract

A deep learning approach to reinforcement learning led to a general learner able to train on visual input to play a variety of arcade games at the human and superhuman levels. Its creators at the Google DeepMind's team called the approach: Deep Q-Network (DQN). We present an extension of DQN by "soft" and "hard" attention mechanisms. Tests of the proposed Deep Attention Recurrent Q-Network (DARQN) algorithm on multiple Atari 2600 games show level of performance superior to that of DQN. Moreover, built-in attention mechanisms allow a direct online monitoring of the training process by highlighting the regions of the game screen the agent is focusing on when making decisions.

7 pages, 5 figures, Deep Reinforcement Learning Workshop, NIPS 2015

References in corpus (3)

• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Recurrent Models of Visual Attention
• Massively Parallel Methods for Deep Reinforcement Learning

Cited by in corpus (25)

• A Brief Survey of Deep Reinforcement Learning
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• End-to-End Deep Reinforcement Learning for Lane Keeping Assist
• ViZDoom Competitions: Playing Doom from Pixels
• Memory-Efficient Backpropagation Through Time
• ACCNet: Actor-Coordinator-Critic Net for "Learning-to-Communicate" with Deep Multi-agent Reinforcement Learning
• Show, Attend and Interact: Perceivable Human-Robot Social Interaction through Neural Attention Q-Network
• Multi-focus Attention Network for Efficient Deep Reinforcement Learning
• Towards Interpretable Reinforcement Learning Using Attention Augmented Agents
• Sample-efficient Reinforcement Learning Representation Learning with Curiosity Contrastive Forward Dynamics Model
• Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation
• An initial attempt of combining visual selective attention with deep reinforcement learning
• The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning
• Attention-Privileged Reinforcement Learning
• A Visual Communication Map for Multi-Agent Deep Reinforcement Learning
• An Attention-Driven Approach of No-Reference Image Quality Assessment
• Are Gradient-based Saliency Maps Useful in Deep Reinforcement Learning?
• Influence-aware Memory Architectures for Deep Reinforcement Learning
• Review, Analysis and Design of a Comprehensive Deep Reinforcement Learning Framework
• Compression and Localization in Reinforcement Learning for ATARI Games
• Pretrained Encoders are All You Need
• Selective Particle Attention: Visual Feature-Based Attention in Deep Reinforcement Learning
• Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning
• Attention or memory? Neurointerpretable agents in space and time
• Neural Networks and Denotation