Dynamic Measurement Scheduling for Event Forecasting using Deep RL

Abstract

Imagine a patient in critical condition. What and when should be measured to forecast detrimental events, especially under the budget constraints? We answer this question by deep reinforcement learning (RL) that jointly minimizes the measurement cost and maximizes predictive gain, by scheduling strategically-timed measurements. We learn our policy to be dynamically dependent on the patient's health history. To scale our framework to exponentially large action space, we distribute our reward in a sequential setting that makes the learning easier. In our simulation, our policy outperforms heuristic-based scheduling with higher predictive gain and lower cost. In a real-world ICU mortality prediction task (MIMIC3), our policies reduce the total number of measurements by $31\%$ or improve predictive gain by a factor of $3$ as compared to physicians, under the off-policy policy evaluation.

ICML 2019

References in corpus (12)

• Dueling Network Architectures for Deep Reinforcement Learning
• Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach
• Explaining Image Classifiers by Counterfactual Generation
• An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis Detection
• A scalable end-to-end Gaussian process adapter for irregularly sampled time series classification
• Representation and Reinforcement Learning for Personalized Glycemic Control in Septic Patients
• Confounding-Robust Policy Improvement
• Treatment-Response Models for Counterfactual Reasoning with Continuous-time, Continuous-valued Interventions
• xGEMs: Generating Examplars to Explain Black-Box Models
• Medical Diagnosis From Laboratory Tests by Combining Generative and Discriminative Learning
• Why Pay More When You Can Pay Less: A Joint Learning Framework for Active Feature Acquisition and Classification
• Active Information Acquisition