Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventions

Abstract

In recent years, reinforcement learning (RL) has acquired a prominent position in health-related sequential decision-making problems, gaining traction as a valuable tool for delivering adaptive interventions (AIs). However, in part due to a poor synergy between the methodological and the applied communities, its real-life application is still limited and its potential is still to be realized. To address this gap, our work provides the first unified technical survey on RL methods, complemented with case studies, for constructing various types of AIs in healthcare. In particular, using the common methodological umbrella of RL, we bridge two seemingly different AI domains, dynamic treatment regimes and just-in-time adaptive interventions in mobile health, highlighting similarities and differences between them and discussing the implications of using RL. Open problems and considerations for future research directions are outlined. Finally, we leverage our experience in designing case studies in both areas to showcase the significant collaborative opportunities between statistical, RL, and healthcare researchers in advancing AIs.

57 pages

References in corpus (11)

• A Contextual-Bandit Approach to Personalized News Article Recommendation
• Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design
• Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
• Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges
• A Reinforcement Learning System to Encourage Physical Activity in Diabetes Patients
• $Q$- and $A$-Learning Methods for Estimating Optimal Dynamic Treatment Regimes
• Q-learning with censored data
• Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Mental Health
• Future-Dependent Value-Based Off-Policy Evaluation in POMDPs
• Estimating Optimal Infinite Horizon Dynamic Treatment Regimes via pT-Learning
• Quasi-optimal Reinforcement Learning with Continuous Actions

Cited by in corpus (1)

• Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimes