A Survey of Deep Reinforcement Learning in Video Games

Abstract

Deep reinforcement learning (DRL) has made great achievements since proposed. Generally, DRL agents receive high-dimensional inputs at each step, and make actions according to deep-neural-network-based policies. This learning mechanism updates the policy to maximize the return with an end-to-end method. In this paper, we survey the progress of DRL methods, including value-based, policy gradient, and model-based algorithms, and compare their main techniques and properties. Besides, DRL plays an important role in game artificial intelligence (AI). We also take a review of the achievements of DRL in various video games, including classical Arcade games, first-person perspective games and multi-agent real-time strategy games, from 2D to 3D, and from single-agent to multi-agent. A large number of video game AIs with DRL have achieved super-human performance, while there are still some challenges in this domain. Therefore, we also discuss some key points when applying DRL methods to this field, including exploration-exploitation, sample efficiency, generalization and transfer, multi-agent learning, imperfect information, and delayed spare rewards, as well as some research directions.

13 pages, 3 figures

References in corpus (11)

• Deep Learning in Neural Networks: An Overview
• A Brief Survey of Deep Reinforcement Learning
• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
• Hybrid Reward Architecture for Reinforcement Learning
• Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learning
• Distributional Reinforcement Learning with Quantile Regression
• Neural Map: Structured Memory for Deep Reinforcement Learning
• Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks
• TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games
• Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning
• Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening

Cited by in corpus (5)

• Predicting Game Difficulty and Churn Without Players
• Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market
• Online reinforcement learning with sparse rewards through an active inference capsule
• Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning
• A Computational Model of Representation Learning in the Brain Cortex, Integrating Unsupervised and Reinforcement Learning