Improved Analysis of the Tsallis-INF Algorithm in Stochastically Constrained Adversarial Bandits and Stochastic Bandits with Adversarial Corruptions

Abstract

We derive improved regret bounds for the Tsallis-INF algorithm of Zimmert and Seldin (2021). We show that in adversarial regimes with a $(\Delta ,C,T)$ self-bounding constraint the algorithm achieves $\mathcal{O}\left(\left({\sum }_{i\ne i^{\ast }}\frac{1}{{\Delta }_i}\right){\log }_{+}\left(\frac{(K-1)T}{{\left({\sum }_{i\ne i^{\ast }}\frac{1}{{\Delta }_i}\right)}^2}\right)+\sqrt{C\left({\sum }_{i\ne i^{\ast }}\frac{1}{{\Delta }_i}\right){\log }_{+}\left(\frac{(K-1)T}{C{\sum }_{i\ne i^{\ast }}\frac{1}{{\Delta }_i}}\right)}\right)$ regret bound, where $T$ is the time horizon, $K$ is the number of arms, ${\Delta }_i$ are the suboptimality gaps, $i^{\ast }$ is the best arm, $C$ is the corruption magnitude, and ${\log }_{+}(x)=\max \left(1,\log x\right)$. The regime includes stochastic bandits, stochastically constrained adversarial bandits, and stochastic bandits with adversarial corruptions as special cases. Additionally, we provide a general analysis, which allows to achieve the same kind of improvement for generalizations of Tsallis-INF to other settings beyond multiarmed bandits.

Published Version in COLT 2021