Latent Bandits Revisited

Abstract

A latent bandit problem is one in which the learning agent knows the arm reward distributions conditioned on an unknown discrete latent state. The primary goal of the agent is to identify the latent state, after which it can act optimally. This setting is a natural midpoint between online and offline learning---complex models can be learned offline with the agent identifying latent state online---of practical relevance in, say, recommender systems. In this work, we propose general algorithms for this setting, based on both upper confidence bounds (UCBs) and Thompson sampling. Our methods are contextual and aware of model uncertainty and misspecification. We provide a unified theoretical analysis of our algorithms, which have lower regret than classic bandit policies when the number of latent states is smaller than actions. A comprehensive empirical study showcases the advantages of our approach.

arXiv comment16 pages, 2 figures

References in corpus (7)

• A Contextual-Bandit Approach to Personalized News Article Recommendation
• Thompson Sampling for Contextual Bandits with Linear Payoffs
• On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems
• Online Clustering of Bandits
• Bounded Regret for Finite-Armed Structured Bandits
• Learning to Optimize Via Posterior Sampling
• Bernoulli Rank-$1$ Bandits for Click Feedback

Cited by in corpus (5)

• Metadata-based Multi-Task Bandits with Bayesian Hierarchical Models
• Non-Stationary Latent Bandits
• Revisiting Clustering of Neural Bandits: Selective Reinitialization for Mitigating Loss of Plasticity
• Online learning in bandits with predicted context
• High Accuracy and Low Regret for User-Cold-Start Using Latent Bandits