Deeply-Debiased Off-Policy Interval Estimation
arXiv:2105.04646
Abstract
Off-policy evaluation learns a target policy's value with a historical dataset generated by a different behavior policy. In addition to a point estimate, many applications would benefit significantly from having a confidence interval (CI) that quantifies the uncertainty of the point estimate. In this paper, we propose a novel deeply-debiasing procedure to construct an efficient, robust, and flexible CI on a target policy's value. Our method is justified by theoretical results and numerical experiments. A Python implementation of the proposed procedure is available at https://github.com/RunzheStat/D2OPE.
References in corpus (9)
- Basic Properties of Strong Mixing Conditions. A Survey and Some Open Questions
- A Theoretical Analysis of Deep Q-Learning
- Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation
- Finite-Sample Analysis for SARSA with Linear Function Approximation
- Minimax Weight and Q-Function Learning for Off-Policy Evaluation
- Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settings
- Inference for Batched Bandits
- Batch Policy Learning in Average Reward Markov Decision Processes
- Accountable Off-Policy Evaluation With Kernel Bellman Statistics