Non-Stationary Latent Bandits
arXiv:2012.00386
Abstract
Users of recommender systems often behave in a non-stationary fashion, due to their evolving preferences and tastes over time. In this work, we propose a practical approach for fast personalization to non-stationary users. The key idea is to frame this problem as a latent bandit, where the prototypical models of user behavior are learned offline and the latent state of the user is inferred online from its interactions with the models. We call this problem a non-stationary latent bandit. We propose Thompson sampling algorithms for regret minimization in non-stationary latent bandits, analyze them, and evaluate them on a real-world dataset. The main strength of our approach is that it can be combined with rich offline-learned models, which can be misspecified, and are subsequently fine-tuned online using posterior sampling. In this way, we naturally combine the strengths of offline and online learning.
arXiv comment15 pages, 4 figures
References in corpus (10)
- Thompson Sampling for Contextual Bandits with Linear Payoffs
- On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems
- Efficient Contextual Bandits in Non-stationary Worlds
- Learning Contextual Bandits in a Non-stationary Environment
- Weighted Linear Bandits for Non-Stationary Environments
- Nearly Optimal Adaptive Procedure with Change Detection for Piecewise-Stationary Bandit
- Thompson Sampling in Switching Environments with Bayesian Online Change Point Detection
- A Simple Approach for Non-stationary Linear Bandits
- Learning to Optimize Via Posterior Sampling
- Latent Bandits Revisited