Metadata-based Multi-Task Bandits with Bayesian Hierarchical Models
arXiv:2108.06422
Abstract
How to explore efficiently is a central problem in multi-armed bandits. In this paper, we introduce the metadata-based multi-task bandit problem, where the agent needs to solve a large number of related multi-armed bandit tasks and can leverage some task-specific features (i.e., metadata) to share knowledge across tasks. As a general framework, we propose to capture task relations through the lens of Bayesian hierarchical models, upon which a Thompson sampling algorithm is designed to efficiently learn task relations, share information, and minimize the cumulative regrets. Two concrete examples for Gaussian bandits and Bernoulli bandits are carefully analyzed. The Bayes regret for Gaussian bandits clearly demonstrates the benefits of information sharing with our algorithm. The proposed method is further supported by extensive experiments.
References in corpus (16)
- VisualBERT: A Simple and Performant Baseline for Vision and Language
- A Survey on Multi-Task Learning
- Further Optimal Regret Bounds for Thompson Sampling
- Online Clustering of Bandits
- A Survey on Contextual Multi-armed Bandits
- A Tutorial on Thompson Sampling
- Latent Bandits Revisited
- PG-TS: Improved Thompson Sampling for Logistic Contextual Bandits
- Online Clustering of Contextual Cascading Bandits
- Empirical Bayes Regret Minimization
- Meta-Learning Bandit Policies by Gradient Ascent
- Adapting to Misspecification in Contextual Bandits with Offline Regression Oracles
- Randomized Allocation with Nonparametric Estimation for Contextual Multi-Armed Bandits with Delayed Rewards
- Meta-Thompson Sampling
- Multinomial Logit Bandit with Linear Utility Functions
- Non-Stationary Latent Bandits