A Direct Approach for Handling Contextual Bandits with Latent State Dynamics

Zhen Li, Gilles Stoltz

We consider a linear contextual bandit model where contexts and rewards are governed by a finite hidden Markov chain. We first revisit the simplified model by Nelson et al. (2022),…