Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks

Abstract

Factorization Machines (FMs) are a supervised learning approach that enhances the linear regression model by incorporating the second-order feature interactions. Despite effectiveness, FM can be hindered by its modelling of all feature interactions with the same weight, as not all feature interactions are equally useful and predictive. For example, the interactions with useless features may even introduce noises and adversely degrade the performance. In this work, we improve FM by discriminating the importance of different feature interactions. We propose a novel model named Attentional Factorization Machine (AFM), which learns the importance of each feature interaction from data via a neural attention network. Extensive experiments on two real-world datasets demonstrate the effectiveness of AFM. Empirically, it is shown on regression task AFM betters FM with a $8.6\%$ relative improvement, and consistently outperforms the state-of-the-art deep learning methods Wide&Deep and DeepCross with a much simpler structure and fewer model parameters. Our implementation of AFM is publicly available at: https://github.com/hexiangnan/attentional_factorization_machine

arXiv comment7 pages, 5 figures

References in corpus (2)

• Wide & Deep Learning for Recommender Systems
• Supervised Discrete Hashing

Cited by in corpus (22)

• Deep Learning based Recommender System: A Survey and New Perspectives
• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• Attentive Aspect Modeling for Review-aware Recommendation
• Self-Attentive Sequential Recommendation
• Deep Session Interest Network for Click-Through Rate Prediction
• Product-based Neural Networks for User Response Prediction over Multi-field Categorical Data
• User Response Prediction in Online Advertising
• Entire Space Multi-Task Modeling via Post-Click Behavior Decomposition for Conversion Rate Prediction
• MARS: Memory Attention-Aware Recommender System
• DeepRec: An Open-source Toolkit for Deep Learning based Recommendation
• Hierarchical Attention Network for Visually-aware Food Recommendation
• Efficient Inner Product Approximation in Hybrid Spaces
• Pairwise Interactive Graph Attention Network for Context-Aware Recommendation
• Sparse Neural Attentive Knowledge-based Models for Grade Prediction
• Operation-aware Neural Networks for User Response Prediction
• Feature Interaction-aware Graph Neural Networks
• Feature Interaction based Neural Network for Click-Through Rate Prediction
• Field-Embedded Factorization Machines for Click-through rate prediction
• Toward a benchmark for CTR prediction in online advertising: datasets, evaluation protocols and perspectives
• The Graph-Based Behavior-Aware Recommendation for Interactive News
• Cross-Modal Alignment with Mixture Experts Neural Network for Intral-City Retail Recommendation
• Attentive Long Short-Term Preference Modeling for Personalized Product Search