A Deep Reinforcement Learning-Based Resource Scheduler for Massive MIMO Networks

Abstract

The large number of antennas in massive MIMO systems allows the base station to communicate with multiple users at the same time and frequency resource with multi-user beamforming. However, highly correlated user channels could drastically impede the spectral efficiency that multi-user beamforming can achieve. As such, it is critical for the base station to schedule a suitable group of users in each time and frequency resource block to achieve maximum spectral efficiency while adhering to fairness constraints among the users. In this paper, we consider the resource scheduling problem for massive MIMO systems with its optimal solution known to be NP-hard. Inspired by recent achievements in deep reinforcement learning (DRL) to solve problems with large action sets, we propose \name{}, a dynamic scheduler for massive MIMO based on the state-of-the-art Soft Actor-Critic (SAC) DRL model and the K-Nearest Neighbors (KNN) algorithm. Through comprehensive simulations using realistic massive MIMO channel models as well as real-world datasets from channel measurement experiments, we demonstrate the effectiveness of our proposed model in various channel conditions. Our results show that our proposed model performs very close to the optimal proportionally fair (Opt-PF) scheduler in terms of spectral efficiency and fairness with more than one order of magnitude lower computational complexity in medium network sizes where Opt-PF is computationally feasible. Our results also show the feasibility and high performance of our proposed scheduler in networks with a large number of users and resource blocks.

arXiv commentIEEE Transactions on Machine Learning in Communications and Networking (TMLCN) 2023

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• A Quantitative Measure Of Fairness And Discrimination For Resource Allocation In Shared Computer Systems
• Prioritized Experience Replay
• Soft Actor-Critic Algorithms and Applications
• Massive MIMO: Ten Myths and One Critical Question
• Deep Reinforcement Learning in Large Discrete Action Spaces
• Soft Actor-Critic for Discrete Action Settings
• Low-Complexity Downlink User Selection for Massive MIMO Systems
• Q-Learning in enormous action spaces via amortized approximate maximization
• Joint QoS-Aware Scheduling and Precoding for Massive MIMO Systems via Deep Reinforcement Learning
• Breaking the Curse of Dimensionality in Multiagent State Space: A Unified Agent Permutation Framework