Ads Recommendation in a Collapsed and Entangled World

Abstract

We present Tencent's ads recommendation system and examine the challenges and practices of learning appropriate recommendation representations. Our study begins by showcasing our approaches to preserving prior knowledge when encoding features of diverse types into embedding representations. We specifically address sequence features, numeric features, and pre-trained embedding features. Subsequently, we delve into two crucial challenges related to feature representation: the dimensional collapse of embeddings and the interest entanglement across different tasks or scenarios. We propose several practical approaches to address these challenges that result in robust and disentangled recommendation representations. We then explore several training techniques to facilitate model optimization, reduce bias, and enhance exploration. Additionally, we introduce three analysis tools that enable us to study feature correlation, dimensional collapse, and interest entanglement. This work builds upon the continuous efforts of Tencent's ads recommendation team over the past decade. It summarizes general design principles and presents a series of readily applicable solutions and analysis tools. The reported performance is based on our online advertising platform, which handles hundreds of billions of requests daily and serves millions of ads to billions of users.

References in corpus (13)

• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
• S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization
• DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
• FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction
• Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising
• Click-Through Rate Prediction in Online Advertising: A Literature Review
• An Embedding Learning Framework for Numerical Features in CTR Prediction
• $F{M}^2$: Field-matrixed Factorization Machines for Recommender Systems
• CL4CTR: A Contrastive Learning Framework for CTR Prediction
• AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning
• Predicting Different Types of Conversions with Multi-Task Learning in Online Advertising
• A Feedback Shift Correction in Predicting Conversion Rates under Delayed Feedback

Cited by in corpus (2)

• Understanding the Ranking Loss for Recommendation with Sparse User Feedback
• MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search