Distributed Exploration in Multi-Armed Bandits

Abstract

We study exploration in Multi-Armed Bandits in a setting where $k$ players collaborate in order to identify an $\epsilon$-optimal arm. Our motivation comes from recent employment of bandit algorithms in computationally intensive, large-scale applications. Our results demonstrate a non-trivial tradeoff between the number of arm pulls required by each of the players, and the amount of communication between them. In particular, our main result shows that by allowing the $k$ players to communicate only once, they are able to learn $\sqrt{k}$ times faster than a single player. That is, distributing learning to $k$ players gives rise to a factor $\sqrt{k}$ parallel speed-up. We complement this result with a lower bound showing this is in general the best possible. On the other extreme, we present an algorithm that achieves the ideal factor $k$ speed-up in learning performance, with communication only logarithmic in $1/\epsilon$.

References in corpus (1)

• Distributed Non-Stochastic Experts

Cited by in corpus (19)

• Game of Thrones: Fully Distributed Learning for Multi-Player Bandits
• Multi-Agent Multi-Armed Bandits with Limited Communication
• Distributed Bandit Learning: Near-Optimal Regret with Efficient Communication
• Collaborative Learning with Limited Interaction: Tight Bounds for Distributed Exploration in Multi-Armed Bandits
• Multiplayer Bandit Learning, from Competition to Cooperation
• Beyond ${\log }^2(T)$ Regret for Decentralized Bandits in Matching Markets
• Robust Multi-Agent Multi-Armed Bandits
• Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximation
• Collaborative Top Distribution Identifications with Limited Interaction
• Linear Bandits with Limited Adaptivity and Learning Distributional Optimal Design
• Exploiting Heterogeneity in Robust Federated Best-Arm Identification
• The Gossiping Insert-Eliminate Algorithm for Multi-Agent Bandits
• On Effective Parallelization of Monte Carlo Tree Search
• Cooperative Stochastic Multi-agent Multi-armed Bandits Robust to Adversarial Corruptions
• Fast Distributed Bandits for Online Recommendation Systems
• Exploration with Limited Memory: Streaming Algorithms for Coin Tossing, Noisy Comparisons, and Multi-Armed Bandits
• Communication Efficient Parallel Reinforcement Learning
• Bandits with Temporal Stochastic Constraints
• Collaborative Pure Exploration in Kernel Bandit