Multi-player Multi-armed Bandits with Collision-Dependent Reward Distributions
arXiv:2106.13669 · doi:10.1109/TSP.2021.3093261
Abstract
We study a new stochastic multi-player multi-armed bandits (MP-MAB) problem, where the reward distribution changes if a collision occurs on the arm. Existing literature always assumes a zero reward for involved players if collision happens, but for applications such as cognitive radio, the more realistic scenario is that collision reduces the mean reward but not necessarily to zero. We focus on the more practical no-sensing setting where players do not perceive collisions directly, and propose the Error-Correction Collision Communication (EC3) algorithm that models implicit communication as a reliable communication over noisy channel problem, for which random coding error exponent is used to establish the optimal regret that no communication protocol can beat. Finally, optimizing the tradeoff between code length and decoding error rate leads to a regret that approaches the centralized MP-MAB regret, which represents a natural lower bound. Experiments with practical error-correction codes on both synthetic and real-world datasets demonstrate the superiority of EC3. In particular, the results show that the choice of coding schemes has a profound impact on the regret performance.
arXiv comment17 pages, 14 figures. Accepted to IEEE Transactions on Signal Processing
References in corpus (9)
- Distributed Learning in Multi-Armed Bandit with Multiple Players
- The KL-UCB Algorithm for Bounded Stochastic Bandits and Beyond
- Distributed Algorithms for Learning and Cognitive Medium Access with Logarithmic Regret
- Decentralized Cooperative Stochastic Bandits
- What Doubling Tricks Can and Can't Do for Multi-Armed Bandits
- A Practical Algorithm for Multiplayer Bandits when Arm Means Vary Among Players
- Multi-Player Bandits: The Adversarial Case
- Decentralized Multi-player Multi-armed Bandits with No Collision Information
- Cooperative and Stochastic Multi-Player Multi-Armed Bandit: Optimal Regret With Neither Communication Nor Collisions