Distributed Deep Q-Learning

Abstract

We propose a distributed deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning. The model is based on the deep Q-network, a convolutional neural network trained with a variant of Q-learning. Its input is raw pixels and its output is a value function estimating future rewards from taking an action given a system state. To distribute the deep Q-network training, we adapt the DistBelief software framework to the context of efficiently training reinforcement learning agents. As a result, the method is completely asynchronous and scales well with the number of machines. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to achieve reasonable success on a simple game with minimal parameter tuning.

Updated figure of distributed deep learning architecture, updated content throughout paper including dealing with minor grammatical issues and highlighting differences of our paper with respect to prior work. arXiv admin note: text overlap with arXiv:1312.5602 by other authors

References in corpus (2)

• Playing Atari with Deep Reinforcement Learning
• Massively Parallel Methods for Deep Reinforcement Learning

Cited by in corpus (7)

• Asynchronous Methods for Deep Reinforcement Learning
• Network Support for High-performance Distributed Machine Learning
• Deep Reinforcement Learning Based Mode Selection and Resource Management for Green Fog Radio Access Networks
• Cache-enabled Wireless Networks with Opportunistic Interference Alignment
• Learning-based Prediction, Rendering and Association Optimization for MEC-enabled Wireless Virtual Reality (VR) Network
• Parallelized Reverse Curriculum Generation
• Human-Level Control without Server-Grade Hardware