The Adaptive Doubly Robust Estimator for Policy Evaluation in Adaptive Experiments and a Paradox Concerning Logging Policy
arXiv:2010.03792
Abstract
The doubly robust (DR) estimator, which consists of two nuisance parameters, the conditional mean outcome and the logging policy (the probability of choosing an action), is crucial in causal inference. This paper proposes a DR estimator for dependent samples obtained from adaptive experiments. To obtain an asymptotically normal semiparametric estimator from dependent samples with non-Donsker nuisance estimators, we propose adaptive-fitting as a variant of sample-splitting. We also report an empirical paradox that our proposed DR estimator tends to show better performances compared to other estimators utilizing the true logging policy. While a similar phenomenon is known for estimators with i.i.d. samples, traditional explanations based on asymptotic efficiency cannot elucidate our case with dependent samples. We confirm this hypothesis through simulation studies.
References in corpus (10)
- A Contextual-Bandit Approach to Personalized News Article Recommendation
- Thompson Sampling for Contextual Bandits with Linear Payoffs
- Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms
- Doubly Robust Policy Evaluation and Learning
- Statistical inference for the mean outcome under a possibly non-unique optimal treatment strategy
- Neural Contextual Bandits with UCB-based Exploration
- Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models
- Off-Policy Evaluation and Learning for External Validity under a Covariate Shift
- Confidence Interval for Off-Policy Evaluation from Dependent Samples via Bandit Algorithm: Approach from Standardized Martingales
- Adaptive Doubly Robust Estimator from Non-stationary Logging Policy under a Convergence of Average Probability