POPCORN: Partially Observed Prediction COnstrained ReiNforcement Learning
arXiv:2001.04032
Abstract
Many medical decision-making tasks can be framed as partially observed Markov decision processes (POMDPs). However, prevailing two-stage approaches that first learn a POMDP and then solve it often fail because the model that best fits the data may not be well suited for planning. We introduce a new optimization objective that (a) produces both high-performing policies and high-quality generative models, even when some observations are irrelevant for planning, and (b) does so in batch off-policy settings that are typical in healthcare, when only retrospective data is available. We demonstrate our approach on synthetic examples and a challenging medical decision-making problem.
arXiv commentAccepted to AISTATS 2020, Palermo, Italy
References in corpus (7)
- Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach
- Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models
- Behaviour Policy Estimation in Off-Policy Policy Evaluation: Calibration Matters
- Policy Optimization via Importance Sampling
- The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making
- Predicting Blood Pressure Response to Fluid Bolus Therapy Using Attention-Based Neural Networks for Clinical Interpretability
- Robust Policy Optimization with Baseline Guarantees
Cited by in corpus (5)
- Learning "What-if" Explanations for Sequential Decision-Making
- Towards Robust Off-Policy Evaluation via Human Inputs
- Trajectory Inspection: A Method for Iterative Clinician-Driven Design of Reinforcement Learning Studies
- Identifying Decision Points for Safe and Interpretable Reinforcement Learning in Hypotension Treatment
- Learning MDPs from Features: Predict-Then-Optimize for Sequential Decision Problems by Reinforcement Learning