139 citations · 493 across the 19 of their papers we have counts for
20 papers
Restless bandits with imperfect binary feedback: PCL-indexability analysis and computation
José Niño-Mora
We study restless bandits with binary latent states and imperfect binary feedback, motivated by opportunistic spectrum access with sensing errors. For the associated belief-state m…
A fast-pivoting algorithm for Whittle's restless bandit index
José Niño-Mora
The Whittle index for restless bandits (two-action semi-Markov decision processes) provides an intuitively appealing optimal policy for controlling a single generic project that ca…
Age of information cost minimization with no buffers, random arrivals and unreliable channels: A PCL-indexability analysis
José Niño-Mora
Over the last decade, the Age of Information has emerged as a key concept and metric for applications where the freshness of sensor-provided data is critical. Limited transmission…
Markovian restless bandits and index policies: A review
José Niño-Mora
The restless multi-armed bandit problem is a paradigmatic modeling framework for optimal dynamic priority allocation in stochastic models of wide-ranging applications that has been…
Multi-gear bandits, partial conservation laws, and indexability
José Niño-Mora
This paper considers what we propose to call multi-gear bandits, which are Markov decision processes modeling a generic dynamic and stochastic project fueled by a single resource a…
A belief-state restless bandit model for treatment adherence: Whittle indexability via partial conservation laws
José Niño-Mora, Ángel Pellitero García
We study clinically motivated capacity-constrained treatment-adherence outreach through a belief-state restless multi-armed bandit model, in which each patient is a partially obser…
Optimal Control of Fluid Restless Multi-armed Bandits: A Machine Learning Approach
Dimitris Bertsimas, Cheol Woo Kim, José Niño-Mora
We present a novel machine learning framework for the optimal control of fluid restless multi-armed bandit problems (FRMABPs) with state equations that are either affine or quadrat…
Multi-Action Restless Bandits with Weakly Coupled Constraints: Simultaneous Learning and Control
Jing Fu, Bill Moran, José Niño-Mora
We study a system with finitely many groups of multi-action bandit processes, each of which is a Markov decision process (MDP) with finite state and action spaces and potentially d…
Non-myopic Beam Scheduling for Multiple Smart Target Tracking in Phased Array Radar Network
Yuhang Hao, Zengfu Wang, José Niño-Mora +3
A smart target, also referred to as a reactive target, can take maneuvering motions to hinder radar tracking. We address beam scheduling for tracking multiple smart targets in phas…
Dynamic priority allocation via restless bandit marginal productivity indices
José Niño-Mora
This paper surveys recent work by the author on the theoretical and algorithmic aspects of restless bandit indexation as well as on its application to a variety of problems involvi…
Dynamic allocation indices for restless projects and queueing admission control: a polyhedral approach
José Niño-Mora
This paper develops a polyhedral approach to the design, analysis, and computation of dynamic allocation indices for scheduling binary-action (engage/rest) Markovian stochastic pro…
A faster index algorithm and a computational study for bandits with switching costs
José Niño-Mora
We address the intractable multi-armed bandit problem with switching costs, for which Asawa and Teneketzis introduced in [M. Asawa and D. Teneketzis. 1996. Multi-armed bandits with…
A fast-pivoting algorithm for the Gittins index and optimal stopping of a Markov chain
José Niño-Mora
This paper presents a new \emph{fast-pivoting} algorithm that computes the Gittins index values of an -state bandit -- in the discounted and undiscounted cases -- by perform…
Computing a classic index for finite-horizon bandits
José Niño-Mora
This paper considers the efficient exact computation of the counterpart of the Gittins index for a finite-horizon discrete-state bandit, which measures for each initial state the a…
Solving Poisson's equation for birth-death chains: Structure, instability, and accurate approximation
José Niño-Mora
Poisson's equation plays a fundamental role as a tool for performance evaluation and optimization of Markov chains. For continuous-time birth-death chains with possibly unbounded t…
Admission and routing of soft real-time jobs to multiclusters: Design and comparison of index policies
José Niño-Mora
Motivated by time-sensitive e-service applications, we consider the design of effective policies in a Markovian model for the dynamic control of both admission and routing of a sin…
Towards minimum loss job routing to parallel heterogeneous multiserver queues via index policies
José Niño-Mora
This paper considers a Markovian model for the optimal dynamic routing of homogeneous traffic to parallel heterogeneous queues, each having its own finite input buffer and server p…
Resource allocation and routing in parallel multi-server queues with abandonments for cloud profit maximization
José Niño-Mora
This paper considers a Markov decision model for profit maximization of a cloud computing service provider catering to customers submitting jobs with firm real-time random deadline…
A verification theorem for threshold-indexability of real-state discounted restless bandits
José Niño-Mora
The Whittle index, which characterizes optimal policies for controlling certain single restless bandit projects (a Markov decision process with two actions: active and passive) is…
A Verification Theorem for Threshold-Indexability of Real-State Discounted Restless Bandits
José Niño-Mora
The Whittle index, which characterizes optimal policies for controlling certain single restless bandit projects (a Markov decision process with two actions: active and passive) is…