Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Abstract

This paper presents a comprehensive literature review on applications of deep reinforcement learning in communications and networking. Modern networks, e.g., Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, deep reinforcement learning, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of deep reinforcement learning from fundamental concepts to advanced models. Then, we review deep reinforcement learning approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks such as 5G and beyond. Furthermore, we present applications of deep reinforcement learning for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying deep reinforcement learning.

arXiv comment37 pages, 13 figures, 6 tables, 174 reference papers

References in corpus (31)

• Continuous control with deep reinforcement learning
• Playing Atari with Deep Reinforcement Learning
• Prioritized Experience Replay
• Dueling Network Architectures for Deep Reinforcement Learning
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Noisy Networks for Exploration
• Deep Reinforcement Learning: An Overview
• Semi-supervised Deep Reinforcement Learning in Support of IoT and Smart City Services
• On Myopic Sensing for Multi-Channel Opportunistic Access: Structure, Optimality, and Performance
• A Distributional Perspective on Reinforcement Learning
• Optimizing User Association and Spectrum Allocation in HetNets: A Utility Perspective
• A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization
• Deep Reinforcement Learning from Self-Play in Imperfect-Information Games
• Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial
• An Efficient Deep Reinforcement Learning Model for Urban Traffic Control
• Experience-driven Networking: A Deep Reinforcement Learning based Approach
• Model-Free Control for Distributed Stream Data Processing using Deep Reinforcement Learning
• Learning-Based Computation Offloading for IoT Devices with Energy Harvesting
• Performance Optimization in Mobile-Edge Computing via Deep Reinforcement Learning
• Towards Cooperation in Sequential Prisoner's Dilemmas: a Deep Multiagent Reinforcement Learning Approach
• Handover Control in Wireless Systems via Asynchronous Multi-User Deep Reinforcement Learning
• Cache-enabled Wireless Networks with Opportunistic Interference Alignment
• Deep Reinforcement Learning for Sponsored Search Real-time Bidding
• Learning Runtime Parameters in Computer Systems with Delayed Experience Injection
• Proactive Resource Management for LTE in Unlicensed Spectrum: A Deep Learning Perspective
• Security in Mobile Edge Caching with Reinforcement Learning
• Deep Reinforcement Learning for Resource Management in Network Slicing
• Reinforcement Learning based Multi-Access Control and Battery Prediction with Energy Harvesting in IoT Systems
• Cache-Enabled Dynamic Rate Allocation via Deep Self-Transfer Reinforcement Learning
• Cell Selection with Deep Reinforcement Learning in Sparse Mobile Crowdsensing
• Echo-Liquid State Deep Learning for $360^{\circ }$ Content Transmission and Caching in Wireless VR Networks with Cellular-Connected UAVs

Cited by in corpus (7)

• DeepRMSA: A Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment in Elastic Optical Networks
• Deep Learning in Mobile and Wireless Networking: A Survey
• Optimizing Throughput Performance in Distributed MIMO Wi-Fi Networks using Deep Reinforcement Learning
• Toward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement Learning
• Intelligent User Association for Symbiotic Radio Networks using Deep Reinforcement Learning
• A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access
• Game Theory and Machine Learning in UAVs-Assisted Wireless Communication Networks: A Survey