Markovian restless bandits and index policies: A review
arXiv:2601.13045 · doi:10.3390/math11071639
Abstract
The restless multi-armed bandit problem is a paradigmatic modeling framework for optimal dynamic priority allocation in stochastic models of wide-ranging applications that has been widely investigated and applied since its inception in a seminal paper by Whittle in the late 1980s. The problem has generated a vast and fast-growing literature from which a significant sample is thematically organized and reviewed in this paper. While the main focus is on priority-index policies due to their intuitive appeal, tractability, asymptotic optimality properties, and often strong empirical performance, other lines of work are also reviewed. Theoretical and algorithmic developments are discussed, along with diverse applications. The main goals are to highlight the remarkable breadth of work that has been carried out on the topic and to stimulate further research in the field.
arXiv comment33 pages
References in corpus (27)
- Dynamic priority allocation via restless bandit marginal productivity indices
- Online Learning of Rested and Restless Bandits
- Dynamic allocation indices for restless projects and queueing admission control: a polyhedral approach
- Asymptotically optimal priority policies for indexable and nonindexable restless bandits
- Computing a classic index for finite-horizon bandits
- On Optimality of Myopic Policy for Restless Multi-armed Bandit Problem with Non i.i.d. Arms and Imperfect Detection
- Whittle index based Q-learning for restless bandits with average reward
- Uncertainty-of-Information Scheduling: A Restless Multi-armed Bandit Framework
- A fast-pivoting algorithm for the Gittins index and optimal stopping of a Markov chain
- A faster index algorithm and a computational study for bandits with switching costs
- Resource allocation and routing in parallel multi-server queues with abandonments for cloud profit maximization
- General notions of indexability for queueing control and asset management
- A Verification Theorem for Threshold-Indexability of Real-State Discounted Restless Bandits
- Q-Learning Lagrange Policies for Multi-Action Restless Bandits
- Admission and routing of soft real-time jobs to multiclusters: Design and comparison of index policies
- Towards minimum loss job routing to parallel heterogeneous multiserver queues via index policies
- Optimal Policies for Observing Time Series and Related Restless Bandit Problems
- Approximations of the Restless Bandit Problem
- Learning in Restless Bandits under Exogenous Global Markov Process
- A fast-pivoting algorithm for Whittle's restless bandit index
- Multi-gear bandits, partial conservation laws, and indexability
- A General Framework of Multi-Armed Bandit Processes by Arm Switch Restrictions
- Efficient Resource Allocation with Fairness Constraints in Restless Multi-Armed Bandits
- When are Kalman-filter restless bandits indexable?
- Networked Restless Multi-Armed Bandits for Mobile Interventions
- NeurWIN: Neural Whittle Index Network For Restless Bandits Via Deep RL
- DeepTOP: Deep Threshold-Optimal Policy for MDPs and RMABs