Whittle Index Based User Association in Dense Millimeter Wave Networks

Abstract

We address the problem of user association in a dense millimeter wave (mmWave) network, in which each arriving user brings a file containing a random number of packets and each time slot is divided into multiple mini-slots. This problem is an instance of the restless multi-armed bandit problem, and is provably hard to solve. Using a technique introduced by Whittle, we relax the hard per-stage constraint that each arriving user must be associated with exactly one mmWave base station (mBS) to a long-term constraint and then use the Lagrangian multiplier technique to convert the problem into an unconstrained problem. This decouples the process governing the system into separate Markov Decision Processes at different mBSs. We prove that the problem is Whittle indexable, present a scheme for computing the Whittle indices of different mBSs, and propose an association scheme under which, each arriving user is associated with the mBS with the smallest value of the Whittle index. Using extensive simulations, we show that the proposed Whittle index based scheme outperforms several user association schemes proposed in prior work in terms of various performance metrics such as average cost, delay, throughput, and Jain's fairness index.

arXiv comment16 pages, 7 figures

References in corpus (7)

• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• User Association in 5G Networks: A Survey and an Outlook
• Exploration in Deep Reinforcement Learning: A Survey
• Asymptotically optimal priority policies for indexable and nonindexable restless bandits
• Joint User Association and Resource Allocation in the Uplink of Heterogeneous Networks
• Whittle index based Q-learning for restless bandits with average reward
• Whittle Index Based User Association in Dense Millimeter Wave Networks

Cited by in corpus (1)

• Whittle Index Based User Association in Dense Millimeter Wave Networks