Neural Factorization Machines for Sparse Predictive Analytics

Abstract

Many predictive tasks of web applications need to model categorical variables, such as user IDs and demographics like genders and occupations. To apply standard machine learning techniques, these categorical predictors are always converted to a set of binary features via one-hot encoding, making the resultant feature vector highly sparse. To learn from such sparse data effectively, it is crucial to account for the interactions between features. Factorization Machines (FMs) are a popular solution for efficiently using the second-order feature interactions. However, FM models feature interactions in a linear way, which can be insufficient for capturing the non-linear and complex inherent structure of real-world data. While deep neural networks have recently been applied to learn non-linear feature interactions in industry, such as the Wide&Deep by Google and DeepCross by Microsoft, the deep structure meanwhile makes them difficult to train. In this paper, we propose a novel model Neural Factorization Machine (NFM) for prediction under sparse settings. NFM seamlessly combines the linearity of FM in modelling second-order feature interactions and the non-linearity of neural network in modelling higher-order feature interactions. Conceptually, NFM is more expressive than FM since FM can be seen as a special case of NFM without hidden layers. Empirical results on two regression tasks show that with one hidden layer only, NFM significantly outperforms FM with a 7.3% relative improvement. Compared to the recent deep learning methods Wide&Deep and DeepCross, our NFM uses a shallower structure but offers better performance, being much easier to train and tune in practice.

10 pages, 8 figures

References in corpus (4)

• BPR: Bayesian Personalized Ranking from Implicit Feedback
• Wide & Deep Learning for Recommender Systems
• Frappe: Understanding the Usage and Perception of Mobile App Recommendations In-The-Wild
• A Mathematical Framework for Feature Selection from Real-World Data with Non-Linear Observations

Cited by in corpus (32)

• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences
• Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems
• Learning to Warm Up Cold Item Embeddings for Cold-start Recommendation with Meta Scaling and Shifting Networks
• $F{M}^2$: Field-matrixed Factorization Machines for Recommender Systems
• Explainable Reasoning over Knowledge Graphs for Recommendation
• Modeling Users' Behavior Sequences with Hierarchical Explainable Network for Cross-domain Fraud Detection
• Predicting Different Types of Conversions with Multi-Task Learning in Online Advertising
• Neural Attentive Session-based Recommendation
• Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction
• TFNet: Multi-Semantic Feature Interaction for CTR Prediction
• Learning from Multi-View Multi-Way Data via Structural Factorization Machines
• Bilinear Graph Neural Network with Neighbor Interactions
• Neural Tensor Factorization
• Sampler Design for Bayesian Personalized Ranking by Leveraging View Data
• Adaptive Factorization Network: Learning Adaptive-Order Feature Interactions
• SAIN: Self-Attentive Integration Network for Recommendation
• Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation
• Fast Matrix Factorization with Non-Uniform Weights on Missing Data
• DeepRec: An Open-source Toolkit for Deep Learning based Recommendation
• MARS: Memory Attention-Aware Recommender System
• Augmenting Recurrent Neural Networks with High-Order User-Contextual Preference for Session-Based Recommendation
• DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving
• Self-Attentive Neural Collaborative Filtering
• Feature Interaction based Neural Network for Click-Through Rate Prediction
• Deep Item-based Collaborative Filtering for Top-N Recommendation
• LSTM Networks for Online Cross-Network Recommendations
• Intrinsic and Extrinsic Factor Disentanglement for Recommendation in Various Context Scenarios
• Attribute-aware Collaborative Filtering: Survey and Classification
• Field-aware Neural Factorization Machine for Click-Through Rate Prediction
• URIR: Recommendation algorithm of user RNN encoder and item encoder based on knowledge graph
• PipAttack: Poisoning Federated Recommender Systems forManipulating Item Promotion