Large-scale Validation of Counterfactual Learning Methods: A Test-Bed

Abstract

The ability to perform effective off-policy learning would revolutionize the process of building better interactive systems, such as search engines and recommendation systems for e-commerce, computational advertising and news. Recent approaches for off-policy evaluation and learning in these settings appear promising. With this paper, we provide real-world data and a standardized test-bed to systematically investigate these algorithms using data from display advertising. In particular, we consider the problem of filling a banner ad with an aggregate of multiple products the user may want to purchase. This paper presents our test-bed, the sanity checks we ran to ensure its validity, and shows results comparing state-of-the-art off-policy learning methods like doubly robust optimization, POEM, and reductions to supervised learning using regression baselines. Our results show experimental evidence that recent off-policy learning methods can improve upon state-of-the-art supervised learning techniques on a large-scale real-world data set.

arXiv comment10 pages, What If workshop NIPS 2016

References in corpus (1)

• Doubly Robust Policy Evaluation and Learning

Cited by in corpus (11)

• KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems
• RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising
• Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects
• Ranking for Relevance and Display Preferences in Complex Presentation Layouts
• Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation
• Imitation-Regularized Offline Learning
• A Field Test of Bandit Algorithms for Recommendations: Understanding the Validity of Assumptions on Human Preferences in Multi-armed Bandits
• Cost-Effective Incentive Allocation via Structured Counterfactual Inference
• The Music Streaming Sessions Dataset
• Uplift Modeling from Separate Labels
• Approaching the Ad Placement Problem with Online Linear Classification: The winning solution to the NIPS'17 Ad Placement Challenge