Efficient kernelized bandit algorithms via exploration distributions

Abstract

We consider a kernelized bandit problem with a compact arm set $X\subset {\mathbb{R}}^d$ and a fixed but unknown reward function $f^{\ast }$ with a finite norm in some Reproducing Kernel Hilbert Space (RKHS). We propose a class of computationally efficient kernelized bandit algorithms, which we call GP-Generic, based on a novel concept: exploration distributions. This class of algorithms includes Upper Confidence Bound-based approaches as a special case, but also allows for a variety of randomized algorithms. With careful choice of exploration distribution, our proposed generic algorithm realizes a wide range of concrete algorithms that achieve $\tilde{O}({\hat{I}}_T^3\sqrt{T})$ regret bounds, where ${\hat{I}}_T^3$ characterizes the RKHS complexity. This matches known results for UCB- and Thompson Sampling-based algorithms; we also show that in practice, randomization can yield better practical results.