Online Regret Bounds for Undiscounted Continuous Reinforcement Learning

Abstract

We derive sublinear regret bounds for undiscounted reinforcement learning in continuous state space. The proposed algorithm combines state aggregation with the use of upper confidence bounds for implementing optimism in the face of uncertainty. Beside the existence of an optimal policy which satisfies the Poisson equation, the only assumptions made are Holder continuity of rewards and transition probabilities.

References in corpus (3)

• REGAL: A Regularization based Algorithm for Reinforcement Learning in Weakly Communicating MDPs
• Regret Bounds for Restless Markov Bandits
• Optimal Regret Bounds for Selecting the State Representation in Reinforcement Learning

Cited by in corpus (1)

• Optimal Regret Bounds for Selecting the State Representation in Reinforcement Learning