Adaptive Estimator Selection for Off-Policy Evaluation

Abstract

We develop a generic data-driven method for estimator selection in off-policy policy evaluation settings. We establish a strong performance guarantee for the method, showing that it is competitive with the oracle estimator, up to a constant factor. Via in-depth case studies in contextual bandits and reinforcement learning, we demonstrate the generality and applicability of the method. We also perform comprehensive experiments, demonstrating the empirical efficacy of our approach and comparing with related approaches. In both case studies, our method compares favorably with existing methods.

arXiv commentFixed some typos. Published in ICML 2020

References in corpus (3)

• Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation
• Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning
• Policy Evaluation and Optimization with Continuous Treatments

Cited by in corpus (5)

• Evaluating the Robustness of Off-Policy Evaluation
• Improving Long-Term Metrics in Recommendation Systems using Short-Horizon Reinforcement Learning
• Optimal Off-Policy Evaluation from Multiple Logging Policies
• Pessimistic Model Selection for Offline Deep Reinforcement Learning
• Optimal Mixture Weights for Off-Policy Evaluation with Multiple Behavior Policies