Lifelong Learning in Multi-Armed Bandits
arXiv:2012.14264
Abstract
Continuously learning and leveraging the knowledge accumulated from prior tasks in order to improve future performance is a long standing machine learning problem. In this paper, we study the problem in the multi-armed bandit framework with the objective to minimize the total regret incurred over a series of tasks. While most bandit algorithms are designed to have a low worst-case regret, we examine here the average regret over bandit instances drawn from some prior distribution which may change over time. We specifically focus on confidence interval tuning of UCB algorithms. We propose a bandit over bandit approach with greedy algorithms and we perform extensive experimental evaluations in both stationary and non-stationary environments. We further apply our solution to the mortal bandit problem, showing empirical improvement over previous work.
References in corpus (13)
- Theoretical Models of Learning to Learn
- Further Optimal Regret Bounds for Thompson Sampling
- Algorithms for multi-armed bandit problems
- A Survey on Practical Applications of Multi-Armed and Contextual Bandits
- Sequential Transfer in Multi-armed Bandit with Finite Set of Models
- Hedging the Drift: Learning to Optimize under Non-Stationarity
- Combinatorial semi-bandit with known covariance
- Learning and Optimization with Seasonal Patterns
- Empirical Bayes Regret Minimization
- Differentiable Bandit Exploration
- A Linear Bandit for Seasonal Environments
- The Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many Arms
- Reducing Exploration of Dying Arms in Mortal Bandits