Multi-facet Contextual Bandits: A Neural Network Perspective
arXiv:2106.03039
Abstract
Contextual multi-armed bandit has shown to be an effective tool in recommender systems. In this paper, we study a novel problem of multi-facet bandits involving a group of bandits, each characterizing the users' needs from one unique aspect. In each round, for the given user, we need to select one arm from each bandit, such that the combination of all arms maximizes the final reward. This problem can find immediate applications in E-commerce, healthcare, etc. To address this problem, we propose a novel algorithm, named MuFasa, which utilizes an assembled neural network to jointly learn the underlying reward functions of multiple bandits. It estimates an Upper Confidence Bound (UCB) linked with the expected reward to balance between exploitation and exploration. Under mild assumptions, we provide the regret analysis of MuFasa. It can achieve the near-optimal regret bound where is the number of bandits and is the number of played rounds. Furthermore, we conduct extensive experiments to show that MuFasa outperforms strong baselines on real-world data sets.
arXiv commentKDD'21, 12 pages
References in corpus (12)
- Neural Tangent Kernel: Convergence and Generalization in Neural Networks
- Thompson Sampling for Contextual Bandits with Linear Payoffs
- HDMI: High-order Deep Multiplex Infomax
- Combinatorial Multi-Armed Bandit and Its Extension to Probabilistically Triggered Arms
- Online Clustering of Bandits
- BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems
- Neural Contextual Bandits with UCB-based Exploration
- Deep Neural Linear Bandits: Overcoming Catastrophic Forgetting through Likelihood Matching
- GAN-based Recommendation with Positive-Unlabeled Sampling
- Generic Outlier Detection in Multi-Armed Bandit
- Deep Co-Attention Network for Multi-View Subspace Learning
- Local Clustering in Contextual Multi-Armed Bandits