Lifelong Learning in Multi-Armed Bandits
arXiv:2012.14264
Abstract
Continuously learning and leveraging the knowledge accumulated from prior tasks in order to improve future performance is a long standing machine learning problem. In this paper, we study the problem in the multi-armed bandit framework with the objective to minimize the total regret incurred over a series of tasks. While most bandit algorithms are designed to have a low worst-case regret, we examine here the average regret over bandit instances drawn from some prior distribution which may change over time. We specifically focus on confidence interval tuning of UCB algorithms. We propose a bandit over bandit approach with greedy algorithms and we perform extensive experimental evaluations in both stationary and non-stationary environments. We further apply our solution to the mortal bandit problem, showing empirical improvement over previous work.
References in corpus (6)
- Theoretical Models of Learning to Learn
- Further Optimal Regret Bounds for Thompson Sampling
- A Survey on Practical Applications of Multi-Armed and Contextual Bandits
- Combinatorial semi-bandit with known covariance
- A Linear Bandit for Seasonal Environments
- Reducing Exploration of Dying Arms in Mortal Bandits