Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback

Abstract

We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to misaligned cost distributions between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approach is both feasible and helpful in practice.

arXiv comment42 pages, 21 figures, ICML 2019

References in corpus (10)

• Thompson Sampling for Contextual Bandits with Linear Payoffs
• Domain Adaptation: Learning Bounds and Algorithms
• Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits
• Contextual Bandit Algorithms with Supervised Learning Guarantees
• Efficient Optimal Learning for Contextual Bandits
• Active Learning from Weak and Strong Labelers
• Online Importance Weight Aware Updates
• A Contextual Bandit Bake-off
• Conservative Contextual Linear Bandits
• Active Learning with Logged Data

Cited by in corpus (4)

• Combining Offline Causal Inference and Online Bandit Learning for Data Driven Decision
• Multitask Bandit Learning Through Heterogeneous Feedback Aggregation
• Bandits with Stochastic Experts: Constant Regret, Empirical Experts and Episodes
• The Label Complexity of Active Learning from Observational Data