Reformulating CTR Prediction: Learning Invariant Feature Interactions for Recommendation

Abstract

Click-Through Rate (CTR) prediction plays a core role in recommender systems, serving as the final-stage filter to rank items for a user. The key to addressing the CTR task is learning feature interactions that are useful for prediction, which is typically achieved by fitting historical click data with the Empirical Risk Minimization (ERM) paradigm. Representative methods include Factorization Machines and Deep Interest Network, which have achieved wide success in industrial applications. However, such a manner inevitably learns unstable feature interactions, i.e., the ones that exhibit strong correlations in historical data but generalize poorly for future serving. In this work, we reformulate the CTR task -- instead of pursuing ERM on historical data, we split the historical data chronologically into several periods (a.k.a, environments), aiming to learn feature interactions that are stable across periods. Such feature interactions are supposed to generalize better to predict future behavior data. Nevertheless, a technical challenge is that existing invariant learning solutions like Invariant Risk Minimization are not applicable, since the click data entangles both environment-invariant and environment-specific correlations. To address this dilemma, we propose Disentangled Invariant Learning (DIL) which disentangles feature embeddings to capture the two types of correlations separately. To improve the modeling efficiency, we further design LightDIL which performs the disentanglement at the higher level of the feature field. Extensive experiments demonstrate the effectiveness of DIL in learning stable feature interactions for CTR. We release the code at https://github.com/zyang1580/DIL.

arXiv comment11 pages, 6 Postscript figures, to be published in SIGIR2023

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Causal Intervention for Leveraging Popularity Bias in Recommendation
• Towards Out-Of-Distribution Generalization: A Survey
• Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising
• Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction
• Causally Regularized Learning with Agnostic Data Selection Bias
• Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems
• Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction
• CausPref: Causal Preference Learning for Out-of-Distribution Recommendation
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction
• Handling Distribution Shifts on Graphs: An Invariance Perspective
• Detecting Arbitrary Order Beneficial Feature Interactions for Recommender Systems
• Meta-Learned Invariant Risk Minimization

Cited by in corpus (7)

• TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation
• Recommendation Unlearning via Influence Function
• User Invariant Preference Learning for Multi-Behavior Recommendation
• GradCraft: Elevating Multi-task Recommendations through Holistic Gradient Crafting
• FedUD: Exploiting Unaligned Data for Cross-Platform Federated Click-Through Rate Prediction
• ELEC: Efficient Large Language Model-Empowered Click-Through Rate Prediction
• I${}^3$-MRec: Invariant Learning with Information Bottleneck for Incomplete Modality Recommendation