Balanced Linear Contextual Bandits
arXiv:1812.06227
Abstract
Contextual bandit algorithms are sensitive to the estimation method of the outcome model as well as the exploration method used, particularly in the presence of rich heterogeneity or complex outcome models, which can lead to difficult estimation problems along the path of learning. We develop algorithms for contextual bandits with linear payoffs that integrate balancing methods from the causal inference literature in their estimation to make it less prone to problems of estimation bias. We provide the first regret bound analyses for linear contextual bandits with balancing and show that our algorithms match the state of the art theoretical guarantees. We demonstrate the strong practical advantage of balanced contextual bandits on a large number of supervised learning datasets and on a synthetic example that simulates model misspecification and prejudice in the initial training data.
arXiv commentAAAI 2019 Oral Presentation. arXiv admin note: substantial text overlap with arXiv:1711.07077
References in corpus (14)
- A Contextual-Bandit Approach to Personalized News Article Recommendation
- Analysis of Thompson Sampling for the multi-armed bandit problem
- Thompson Sampling for Contextual Bandits with Linear Payoffs
- Doubly Robust Policy Evaluation and Learning
- Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits
- Contextual Bandits with Similarity Information
- Nonparametric Bandits with Covariates
- Making Contextual Decisions with Low Technical Debt
- Causal Bandits: Learning Good Interventions via Causal Inference
- A Contextual Bandit Bake-off
- An Actor-Critic Contextual Bandit Algorithm for Personalized Mobile Health Interventions
- Estimation Considerations in Contextual Bandits
- Counterfactual Estimation and Optimization of Click Metrics for Search Engines
- Accurate Inference for Adaptive Linear Models