Dynamic Measurement Scheduling for Adverse Event Forecasting using Deep RL
arXiv:1812.00268
Abstract
Current clinical practice to monitor patients' health follows either regular or heuristic-based lab test (e.g. blood test) scheduling. Such practice not only gives rise to redundant measurements accruing cost, but may even lead to unnecessary patient discomfort. From the computational perspective, heuristic-based test scheduling might lead to reduced accuracy of clinical forecasting models. Computationally learning an optimal clinical test scheduling and measurement collection, is likely to lead to both, better predictive models and patient outcome improvement. We address the scheduling problem using deep reinforcement learning (RL) to achieve high predictive gain and low measurement cost, by scheduling fewer, but strategically timed tests. We first show that in the simulation our policy outperforms heuristic-based measurement scheduling with higher predictive gain or lower cost measured by accumulated reward. We then learn a scheduling policy for mortality forecasting in the real-world clinical dataset (MIMIC3), our learned policy is able to provide useful clinical insights. To our knowledge, this is the first RL application on multi-measurement scheduling problem in the clinical setting.
arXiv commentMachine Learning for Health (ML4H) Workshop at NeurIPS 2018 arXiv:1811.07216
References in corpus (9)
- Dueling Network Architectures for Deep Reinforcement Learning
- Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach
- Doubly Robust Off-policy Value Evaluation for Reinforcement Learning
- An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis Detection
- Representation and Reinforcement Learning for Personalized Glycemic Control in Septic Patients
- Adversarial Training for Disease Prediction from Electronic Health Records with Missing Data
- Medical Diagnosis From Laboratory Tests by Combining Generative and Discriminative Learning
- Active Information Acquisition
- Why Pay More When You Can Pay Less: A Joint Learning Framework for Active Feature Acquisition and Classification