UCB Exploration via Q-Ensembles

Abstract

We show how an ensemble of $Q^{\ast }$-functions can be leveraged for more effective exploration in deep reinforcement learning. We build on well established algorithms from the bandit setting, and adapt them to the $Q$-learning setting. We propose an exploration strategy based on upper-confidence bounds (UCB). Our experiments show significant gains on the Atari benchmark.

References in corpus (1)

• Model-Based Bayesian Exploration

Cited by in corpus (2)

• Off-Policy Actor-Critic in an Ensemble: Achieving Maximum General Entropy and Effective Environment Exploration in Deep Reinforcement Learning
• Variational Bayes: A report on approaches and applications