Generalized Thompson Sampling for Contextual Bandits

Abstract

Thompson Sampling, one of the oldest heuristics for solving multi-armed bandits, has recently been shown to demonstrate state-of-the-art performance. The empirical success has led to great interests in theoretical understanding of this heuristic. In this paper, we approach this problem in a way very different from existing efforts. In particular, motivated by the connection between Thompson Sampling and exponentiated updates, we propose a new family of algorithms called Generalized Thompson Sampling in the expert-learning framework, which includes Thompson Sampling as a special case. Similar to most expert-learning algorithms, Generalized Thompson Sampling uses a loss function to adjust the experts' weights. General regret bounds are derived, which are also instantiated to two important loss functions: square loss and logarithmic loss. In contrast to existing bounds, our results apply to quite general contextual bandits. More importantly, they quantify the effect of the "prior" distribution on the regret bounds.

References in corpus (3)

• Analysis of Thompson Sampling for the multi-armed bandit problem
• Thompson Sampling for Contextual Bandits with Linear Payoffs
• Further Optimal Regret Bounds for Thompson Sampling

Cited by in corpus (8)

• Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits
• An Information-Theoretic Analysis of Thompson Sampling
• Online Stochastic Linear Optimization under One-bit Feedback
• Latent Contextual Bandits and their Application to Personalized Recommendations for New Users
• Learning to Optimize Via Posterior Sampling
• On the Prior Sensitivity of Thompson Sampling
• Personalized Web Search
• A Note on Information-Directed Sampling and Thompson Sampling