$Q$- and $A$-Learning Methods for Estimating Optimal Dynamic Treatment Regimes

Abstract

In clinical practice, physicians make a series of treatment decisions over the course of a patient's disease based on his/her baseline and evolving characteristics. A dynamic treatment regime is a set of sequential decision rules that operationalizes this process. Each rule corresponds to a decision point and dictates the next treatment action based on the accrued information. Using existing data, a key goal is estimating the optimal regime, that, if followed by the patient population, would yield the most favorable outcome on average. Q- and A-learning are two main approaches for this purpose. We provide a detailed account of these methods, study their performance, and illustrate them using data from a depression study.

Published in at http://dx.doi.org/10.1214/13-STS450 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

References in corpus (1)

• Statistical Inference in Dynamic Treatment Regimes

Cited by in corpus (8)

• Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventions
• Causal Etiology of the Research of James M. Robins
• Evaluating the Effectiveness of Personalized Medicine with Software
• Robust Estimation of Heterogeneous Treatment Effects using Electronic Health Record Data
• Efficient adjustment sets in causal graphical models with hidden variables
• Relative Sparsity for Medical Decision Problems
• Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimes
• Q-learning in Dynamic Treatment Regimes with Misclassified Binary Outcome