On Gradient Boosted Decision Trees and Neural Rankers: A Case-Study on Short-Video Recommendations at ShareChat

Abstract

Practitioners who wish to build real-world applications that rely on ranking models, need to decide which modelling paradigm to follow. This is not an easy choice to make, as the research literature on this topic has been shifting in recent years. In particular, whilst Gradient Boosted Decision Trees (GBDTs) have reigned supreme for more than a decade, the flexibility of neural networks has allowed them to catch up, and recent works report accuracy metrics that are on par. Nevertheless, practical systems require considerations beyond mere accuracy metrics to decide on a modelling approach. This work describes our experiences in balancing some of the trade-offs that arise, presenting a case study on a short-video recommendation application. We highlight (1) neural networks' ability to handle large training data size, user- and item-embeddings allows for more accurate models than GBDTs in this setting, and (2) because GBDTs are less reliant on specialised hardware, they can provide an equally accurate model at a lower cost. We believe these findings are of relevance to researchers in both academia and industry, and hope they can inspire practitioners who need to make similar modelling choices in the future.

arXiv commentAppearing in the Industry Track Proceedings of the Forum for Information Retrieval Evaluation (FIRE '23)

References in corpus (15)

• A Comparative Analysis of XGBoost
• CatBoost: gradient boosting with categorical features support
• DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
• Benchmarking TPU, GPU, and CPU Platforms for Deep Learning
• On Application of Learning to Rank for E-Commerce Search
• When Do Neural Nets Outperform Boosted Trees on Tabular Data?
• Multi-Task Fusion via Reinforcement Learning for Long-Term User Satisfaction in Recommender Systems
• Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time
• MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask
• Monolith: Real Time Recommendation System With Collisionless Embedding Table
• Visual Discovery at Pinterest
• Related Pins at Pinterest: The Evolution of a Real-World Recommender System
• StochasticRank: Global Optimization of Scale-Free Discrete Functions
• Which Tricks Are Important for Learning to Rank?
• On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-$n$ Recommendation