Meta-learning with Stochastic Linear Bandits

Abstract

We investigate meta-learning procedures in the setting of stochastic linear bandits tasks. The goal is to select a learning algorithm which works well on average over a class of bandits tasks, that are sampled from a task-distribution. Inspired by recent work on learning-to-learn linear regression, we consider a class of bandit algorithms that implement a regularized version of the well-known OFUL algorithm, where the regularization is a square euclidean distance to a bias vector. We first study the benefit of the biased OFUL algorithm in terms of regret minimization. We then propose two strategies to estimate the bias within the learning-to-learn setting. We show both theoretically and experimentally, that when the number of tasks grows and the variance of the task-distribution is small, our strategies have a significant advantage over learning the tasks in isolation.

References in corpus (7)

• A Contextual-Bandit Approach to Personalized News Article Recommendation
• The Benefit of Multitask Representation Learning
• Online Clustering of Bandits
• Sequential Transfer in Multi-armed Bandit with Finite Set of Models
• Incremental Learning-to-Learn with Statistical Guarantees
• Efficient Linear Bandits through Matrix Sketching
• Stochastic Bandits with Delay-Dependent Payoffs

Cited by in corpus (10)

• Continuous Meta-Learning without Tasks
• Meta-Learning Bandit Policies by Gradient Ascent
• Meta-Thompson Sampling
• Hierarchical Bayesian Bandits
• No Regrets for Learning the Prior in Bandits
• Transfer Learning in Bandits with Latent Continuity
• Metalearning Linear Bandits by Prior Update
• Learning to Actively Learn: A Robust Approach
• Online Parameter-Free Learning of Multiple Low Variance Tasks
• Online Model Selection: a Rested Bandit Formulation