Fighting Bandits with a New Kind of Smoothness

Abstract

We define a novel family of algorithms for the adversarial multi-armed bandit problem, and provide a simple analysis technique based on convex smoothing. We prove two main results. First, we show that regularization via the \emph{Tsallis entropy}, which includes EXP3 as a special case, achieves the $\Theta (\sqrt{TN})$ minimax regret. Second, we show that a wide class of perturbation methods achieve a near-optimal regret as low as $O(\sqrt{TN\log N})$ if the perturbation distribution has a bounded hazard rate. For example, the Gumbel, Weibull, Frechet, Pareto, and Gamma distributions all satisfy this key property.

arXiv commentIn Proceedings of NIPS, 2015

References in corpus (3)

• Efficient learning by implicit exploration in bandit problems with side observations
• Online Linear Optimization via Smoothing
• Prediction by Random-Walk Perturbation

Cited by in corpus (11)

• Online Learning: A Modern Introduction Using Convex Optimization
• Tsallis-INF: An Optimal Algorithm for Stochastic and Adversarial Bandits
• Fair Contextual Multi-Armed Bandits: Theory and Experiments
• Optimal Algorithms for Stochastic Multi-Armed Bandits with Heavy Tailed Rewards
• Efficient Competitions and Online Learning with Strategic Forecasters
• On the Optimality of Perturbations in Stochastic and Adversarial Multi-armed Bandit Problems
• Randomized Exploration for Non-Stationary Stochastic Linear Bandits
• Convex Regularization in Monte-Carlo Tree Search
• Banker Online Mirror Descent
• Adaptive Online Learning for Gradient-Based Optimizers
• Scale-Free Adversarial Multi-Armed Bandit with Arbitrary Feedback Delays