Scalable Coordinated Exploration in Concurrent Reinforcement Learning

Abstract

We consider a team of reinforcement learning agents that concurrently operate in a common environment, and we develop an approach to efficient coordinated exploration that is suitable for problems of practical scale. Our approach builds on seed sampling (Dimakopoulou and Van Roy, 2018) and randomized value function learning (Osband et al., 2016). We demonstrate that, for simple tabular contexts, the approach is competitive with previously proposed tabular model learning methods (Dimakopoulou and Van Roy, 2018). With a higher-dimensional problem and a neural network value function representation, the approach learns quickly with far fewer agents than alternative exploration schemes.

NIPS 2018

References in corpus (5)

• DeepMind Control Suite
• Randomized Prior Functions for Deep Reinforcement Learning
• A Tutorial on Thompson Sampling
• On Optimistic versus Randomized Exploration in Reinforcement Learning
• Posterior Sampling for Reinforcement Learning Without Episodes

Cited by in corpus (4)

• Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
• Collaborative Top Distribution Identifications with Limited Interaction
• Online Sub-Sampling for Reinforcement Learning with General Function Approximation
• PAC Guarantees for Cooperative Multi-Agent Reinforcement Learning with Restricted Communication