Deep Reinforcement Learning for Radio Resource Allocation and Management in Next Generation Heterogeneous Wireless Networks: A Survey

Abstract

Next generation wireless networks are expected to be extremely complex due to their massive heterogeneity in terms of the types of network architectures they incorporate, the types and numbers of smart IoT devices they serve, and the types of emerging applications they support. In such large-scale and heterogeneous networks (HetNets), radio resource allocation and management (RRAM) becomes one of the major challenges encountered during system design and deployment. In this context, emerging Deep Reinforcement Learning (DRL) techniques are expected to be one of the main enabling technologies to address the RRAM in future wireless HetNets. In this paper, we conduct a systematic in-depth, and comprehensive survey of the applications of DRL techniques in RRAM for next generation wireless networks. Towards this, we first overview the existing traditional RRAM methods and identify their limitations that motivate the use of DRL techniques in RRAM. Then, we provide a comprehensive review of the most widely used DRL algorithms to address RRAM problems, including the value- and policy-based algorithms. The advantages, limitations, and use-cases for each algorithm are provided. We then conduct a comprehensive and in-depth literature review and classify existing related works based on both the radio resources they are addressing and the type of wireless networks they are investigating. To this end, we carefully identify the types of DRL algorithms utilized in each related work, the elements of these algorithms, and the main findings of each related work. Finally, we highlight important open challenges and provide insights into several future research directions in the context of DRL-based RRAM. This survey is intentionally designed to guide and stimulate more research endeavors towards building efficient and fine-grained DRL-based RRAM schemes for future wireless networks.

References in corpus (23)

• Continuous control with deep reinforcement learning
• A Brief Survey of Deep Reinforcement Learning
• Addressing Function Approximation Error in Actor-Critic Methods
• Prioritized Experience Replay
• Fractional Programming for Communication Systems--Part I: Power Control and Beamforming
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management
• Deep Recurrent Q-Learning for Partially Observable MDPs
• Artificial Intelligence-Enabled Intelligent 6G Networks
• Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks
• Distributed Prioritized Experience Replay
• On Optimal Power Allocation for Downlink Non-Orthogonal Multiple Access Systems
• A Survey of Rate-optimal Power Domain NOMA with Enabling Technologies of Future Wireless Networks
• Distributed Distributional Deterministic Policy Gradients
• Intelligent Power Control for Spectrum Sharing in Cognitive Radios: A Deep Reinforcement Learning Approach
• A Multi-Objective Deep Reinforcement Learning Framework
• Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
• PrecoderNet: Hybrid Beamforming for Millimeter Wave Systems with Deep Reinforcement Learning
• Optimization for Reinforcement Learning: From Single Agent to Cooperative Agents
• Reinforcement Learning Based Vehicle-cell Association Algorithm for Highly Mobile Millimeter Wave Communication
• Search Algorithms for Automated Hyper-Parameter Tuning
• Deep Reinforcement Learning Architecture for Continuous Power Allocation in High Throughput Satellites
• Challenges of Applying Deep Reinforcement Learning in Dynamic Dispatching

Cited by in corpus (1)

• Towards Practical Operation of Deep Reinforcement Learning Agents in Real-World Network Management at Open RAN Edges