An Asymptotically Optimal Index Policy for Finite-Horizon Restless Bandits

Abstract

We consider restless multi-armed bandit (RMAB) with a finite horizon and multiple pulls per period. Leveraging the Lagrangian relaxation, we approximate the problem with a collection of single arm problems. We then propose an index-based policy that uses optimal solutions of the single arm problems to index individual arms, and offer a proof that it is asymptotically optimal as the number of arms tends to infinity. We also use simulation to show that this index-based policy performs better than the state-of-art heuristics in various problem settings.

References in corpus (1)

• Weakly Coupled Dynamic Program: Information and Lagrangian Relaxations

Cited by in corpus (2)

• Restless Bandits with Many Arms: Beating the Central Limit Theorem
• Reinforcement Learning for Finite-Horizon Restless Multi-Armed Multi-Action Bandits