$F{M}^2$: Field-matrixed Factorization Machines for Recommender Systems

Abstract

Click-through rate (CTR) prediction plays a critical role in recommender systems and online advertising. The data used in these applications are multi-field categorical data, where each feature belongs to one field. Field information is proved to be important and there are several works considering fields in their models. In this paper, we proposed a novel approach to model the field information effectively and efficiently. The proposed approach is a direct improvement of FwFM, and is named as Field-matrixed Factorization Machines (FmFM, or $F{M}^2$). We also proposed a new explanation of FM and FwFM within the FmFM framework, and compared it with the FFM. Besides pruning the cross terms, our model supports field-specific variable dimensions of embedding vectors, which acts as soft pruning. We also proposed an efficient way to minimize the dimension while keeping the model performance. The FmFM model can also be optimized further by caching the intermediate vectors, and it only takes thousands of floating-point operations (FLOPs) to make a prediction. Our experiment results show that it can out-perform the FFM, which is more complex. The FmFM model's performance is also comparable to DNN models which require much more FLOPs in runtime.

In Proceedings of the Web Conference 2021 (WWW 2021), April 19-23, 2021, Ljubljana, Slovenia. 10 pages

References in corpus (8)

• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
• DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
• Neural Factorization Machines for Sparse Predictive Analytics
• Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising
• Deep & Cross Network for Ad Click Predictions
• Rocket Launching: A Universal and Efficient Framework for Training Well-performing Light Net
• Predicting Different Types of Conversions with Multi-Task Learning in Online Advertising

Cited by in corpus (13)

• Click-Through Rate Prediction in Online Advertising: A Literature Review
• SAR-Net: A Scenario-Aware Ranking Network for Personalized Fair Recommendation in Hundreds of Travel Scenarios
• CL4CTR: A Contrastive Learning Framework for CTR Prediction
• EulerNet: Adaptive Feature Interaction Learning via Euler's Formula for CTR Prediction
• CausPref: Causal Preference Learning for Out-of-Distribution Recommendation
• From Data to Decisions: The Transformational Power of Machine Learning in Business Recommendations
• Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation
• Ads Recommendation in a Collapsed and Entangled World
• Understanding the Ranking Loss for Recommendation with Sparse User Feedback
• GraphFM: Graph Factorization Machines for Feature Interaction Modeling
• Temporal Interest Network for User Response Prediction
• Toward a benchmark for CTR prediction in online advertising: datasets, evaluation protocols and perspectives
• Dual-Stream MLP is All You Need for CTR Prediction