Taming Non-stationary Bandits: A Bayesian Approach
arXiv:1707.09727
Abstract
We consider the multi armed bandit problem in non-stationary environments. Based on the Bayesian method, we propose a variant of Thompson Sampling which can be used in both rested and restless bandit scenarios. Applying discounting to the parameters of prior distribution, we describe a way to systematically reduce the effect of past observations. Further, we derive the exact expression for the probability of picking sub-optimal arms. By increasing the exploitative value of Bayes' samples, we also provide an optimistic version of the algorithm. Extensive empirical analysis is conducted under various scenarios to validate the utility of proposed algorithms. A comparison study with various state-of-the-arm algorithms is also included.
arXiv commentSubmitted to NIPS 2017
References in corpus (4)
Cited by in corpus (15)
- An empirical evaluation of active inference in multi-armed bandits
- Hedging the Drift: Learning to Optimize under Non-Stationarity
- Learning and Optimization with Seasonal Patterns
- Adapting Behaviour for Learning Progress
- Nonstationary Stochastic Multiarmed Bandits: UCB Policies and Minimax Regret
- SLOPT: Bandit Optimization Framework for Mutation-Based Fuzzing
- AutoSeM: Automatic Task Selection and Mixing in Multi-Task Learning
- Blending Search and Discovery: Tag-Based Query Refinement with Contextual Reinforcement Learning
- Distribution-dependent and Time-uniform Bounds for Piecewise i.i.d Bandits
- Boltzmann Exploration Expectation-Maximisation
- Existence conditions for hidden feedback loops in online recommender systems
- Accelerated learning from recommender systems using multi-armed bandit
- On Limited-Memory Subsampling Strategies for Bandits
- On Adaptive Estimation for Dynamic Bernoulli Bandits
- Odds-Ratio Thompson Sampling to Control for Time-Varying Effect