Combining Offline Causal Inference and Online Bandit Learning for Data Driven Decision
arXiv:2001.05699
Abstract
A fundamental question for companies with large amount of logged data is: How to use such logged data together with incoming streaming data to make good decisions? Many companies currently make decisions via online A/B tests, but wrong decisions during testing hurt users' experiences and cause irreversible damage. A typical alternative is offline causal inference, which analyzes logged data alone to make decisions. However, these decisions are not adaptive to the new incoming data, and so a wrong decision will continuously hurt users' experiences. To overcome the aforementioned limitations, we propose a framework to unify offline causal inference algorithms (e.g., weighting, matching) and online learning algorithms (e.g., UCB, LinUCB). We propose novel algorithms and derive bounds on the decision accuracy via the notion of "regret". We derive the first upper regret bound for forest-based online bandit algorithms. Experiments on two real datasets show that our algorithms outperform other algorithms that use only logged data or online feedbacks, or algorithms that do not use the data properly.
arXiv comment27 pages, 35 figures, 4 tables
References in corpus (12)
- Matching Methods for Causal Inference: A Review and a Look Forward
- Analysis of Thompson Sampling for the multi-armed bandit problem
- Doubly Robust Policy Evaluation and Learning
- Algorithms for multi-armed bandit problems
- Counterfactual Risk Minimization: Learning from Logged Bandit Feedback
- Causal Bandits: Learning Good Interventions via Causal Inference
- Balanced Policy Evaluation and Learning
- The Deconfounded Recommender: A Causal Inference Approach to Recommendation
- Estimation Considerations in Contextual Bandits
- An Information-Theoretic Analysis for Thompson Sampling with Many Actions
- Bounded regret in stochastic multi-armed bandits
- Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback