Bandit Algorithms for Precision Medicine

Abstract

The Oxford English Dictionary defines precision medicine as "medical care designed to optimize efficiency or therapeutic benefit for particular groups of patients, especially by using genetic or molecular profiling." It is not an entirely new idea: physicians from ancient times have recognized that medical treatment needs to consider individual variations in patient characteristics. However, the modern precision medicine movement has been enabled by a confluence of events: scientific advances in fields such as genetics and pharmacology, technological advances in mobile devices and wearable sensors, and methodological advances in computing and data sciences. This chapter is about bandit algorithms: an area of data science of special relevance to precision medicine. With their roots in the seminal work of Bellman, Robbins, Lai and others, bandit algorithms have come to occupy a central place in modern data science ( Lattimore and Szepesvari, 2020). Bandit algorithms can be used in any situation where treatment decisions need to be made to optimize some health outcome. Since precision medicine focuses on the use of patient characteristics to guide treatment, contextual bandit algorithms are especially useful since they are designed to take such information into account. The role of bandit algorithms in areas of precision medicine such as mobile health and digital phenotyping has been reviewed before (Tewari and Murphy, 2017; Rabbi et al., 2019). Since these reviews were published, bandit algorithms have continued to find uses in mobile health and several new topics have emerged in the research on bandit algorithms. This chapter is written for quantitative researchers in fields such as statistics, machine learning, and operations research who might be interested in knowing more about the algorithmic and mathematical details of bandit algorithms that have been used in mobile health.

To appear as a chapter in the Handbook of Statistical Methods for Precision Medicine edited by Tianxi Cai, Bibhas Chakraborty, Eric Laber, Erica Moodie, and Mark van der Laan

References in corpus (36)

• A Contextual-Bandit Approach to Personalized News Article Recommendation
• Analysis of Thompson Sampling for the multi-armed bandit problem
• Counterfactual Fairness
• Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
• Fairness of Exposure in Rankings
• Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms
• Further Optimal Regret Bounds for Thompson Sampling
• Doubly Robust Policy Evaluation and Learning
• Fairness in Learning: Classic and Contextual Bandits
• The Benefit of Multitask Representation Learning
• Knapsack based Optimal Policies for Budget-Limited Multi-Armed Bandits
• Online Clustering of Bandits
• Online Learning with an Unknown Fairness Metric
• Causal Bandits: Learning Good Interventions via Causal Inference
• Algorithms with Logarithmic or Sublinear Regret for Constrained Contextual Bandits
• Linear Contextual Bandits with Knapsacks
• Nearly Optimal Adaptive Procedure with Change Detection for Piecewise-Stationary Bandit
• Thompson Sampling for Budgeted Multi-armed Bandits
• Multi-Task Learning for Contextual Bandits
• Optimally Confident UCB: Improved Regret for Finite-Armed Bandits
• Conservative Bandits
• Identifying Best Interventions through Online Importance Sampling
• Budget-Constrained Multi-Armed Bandits with Multiple Plays
• Non-stationary Reinforcement Learning without Prior Knowledge: An Optimal Black-box Approach
• Power Constrained Bandits
• Sparsity regret bounds for individual sequences in online linear regression
• High-Dimensional Sparse Linear Bandits
• Improving offline evaluation of contextual bandit algorithms via bootstrapping techniques
• Fairness of Exposure in Stochastic Bandits
• MOTS: Minimax Optimal Thompson Sampling
• The Elliptical Potential Lemma Revisited
• Causal Bandits with Unknown Graph Structure
• What You See May Not Be What You Get: UCB Bandit Algorithms Robust to ε-Contamination
• Group Fairness in Bandit Arm Selection
• Doubly robust Thompson sampling for linear payoffs
• A Biologically Plausible Benchmark for Contextual Bandit Algorithms in Precision Oncology Using in vitro Data

Cited by in corpus (1)

• Doubly Robust Interval Estimation for Optimal Policy Evaluation in Online Learning