Accelerated learning from recommender systems using multi-armed bandit

Abstract

Recommendation systems are a vital component of many online marketplaces, where there are often millions of items to potentially present to users who have a wide variety of wants or needs. Evaluating recommender system algorithms is a hard task, given all the inherent bias in the data, and successful companies must be able to rapidly iterate on their solution to maintain their competitive advantage. The gold standard for evaluating recommendation algorithms has been the A/B test since it is an unbiased way to estimate how well one or more algorithms compare in the real world. However, there are a number of issues with A/B testing that make it impractical to be the sole method of testing, including long lead time, and high cost of exploration. We argue that multi armed bandit (MAB) testing as a solution to these issues. We showcase how we implemented a MAB solution as an extra step between offline and online A/B testing in a production system. We present the result of our experiment and compare all the offline, MAB, and online A/B tests metrics for our use case.

References in corpus (32)

• Analysis of Thompson Sampling for the multi-armed bandit problem
• Distributed Learning in Multi-Armed Bandit with Multiple Players
• The KL-UCB Algorithm for Bounded Stochastic Bandits and Beyond
• Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits
• Wide & Deep Learning for Recommender Systems
• On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems
• An Efficient Bandit Algorithm for Realtime Multivariate Optimization
• Direct Optimization of Ranking Measures
• On the Difficulty of Evaluating Baselines: A Study on Recommender Systems
• Minimal Exploration in Structured Stochastic Bandits
• A Survey on Session-based Recommender Systems
• Optimal Regret Analysis of Thompson Sampling in Stochastic Multi-armed Bandit Problem with Multiple Plays
• Nearly Optimal Adaptive Procedure with Change Detection for Piecewise-Stationary Bandit
• Taming Non-stationary Bandits: A Bayesian Approach
• Word2Vec applied to Recommendation: Hyperparameters Matter
• An Improved Parametrization and Analysis of the EXP3++ Algorithm for Stochastic and Adversarial Bandits
• Optimally Confident UCB: Improved Regret for Finite-Armed Bandits
• Bandit-Based Model Selection for Deformable Object Manipulation
• Boltzmann Exploration Done Right
• Regret Analysis of the Anytime Optimally Confident UCB Algorithm
• SIC-MMAB: Synchronisation Involves Communication in Multiplayer Multi-Armed Bandits
• KL-UCB-switch: optimal regret bounds for stochastic bandits from both a distribution-dependent and a distribution-free viewpoints
• Tsallis-INF: An Optimal Algorithm for Stochastic and Adversarial Bandits
• Concurrent bandits and cognitive radio networks
• On Search Engine Evaluation Metrics
• Corralling a Band of Bandit Algorithms
• On the Optimality of Perturbations in Stochastic and Adversarial Multi-armed Bandit Problems
• A Simple Deep Personalized Recommendation System
• Sparse Stochastic Bandits
• Learning to Use Learners' Advice
• Productization Challenges of Contextual Multi-Armed Bandits
• UCBoost: A Boosting Approach to Tame Complexity and Optimality for Stochastic Bandits