Context-lumpable stochastic bandits

Chung-Wei Lee, Qinghua Liu, Yasin Abbasi-Yadkori +3

We consider a contextual bandit problem with $S$ contexts and $K$ actions. In each round $t=1,2,\dots$, the learner observes a random context and chooses an action based on its pas…