AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks

Abstract

Click-through rate (CTR) prediction, which aims to predict the probability of a user clicking on an ad or an item, is critical to many online applications such as online advertising and recommender systems. The problem is very challenging since (1) the input features (e.g., the user id, user age, item id, item category) are usually sparse and high-dimensional, and (2) an effective prediction relies on high-order combinatorial features (\textit{a.k.a.} cross features), which are very time-consuming to hand-craft by domain experts and are impossible to be enumerated. Therefore, there have been efforts in finding low-dimensional representations of the sparse and high-dimensional raw features and their meaningful combinations. In this paper, we propose an effective and efficient method called the \emph{AutoInt} to automatically learn the high-order feature interactions of input features. Our proposed algorithm is very general, which can be applied to both numerical and categorical input features. Specifically, we map both the numerical and categorical features into the same low-dimensional space. Afterwards, a multi-head self-attentive neural network with residual connections is proposed to explicitly model the feature interactions in the low-dimensional space. With different layers of the multi-head self-attentive neural networks, different orders of feature combinations of input features can be modeled. The whole model can be efficiently fit on large-scale raw data in an end-to-end fashion. Experimental results on four real-world datasets show that our proposed approach not only outperforms existing state-of-the-art approaches for prediction but also offers good explainability. Code is available at: \url{https://github.com/DeepGraphLearning/RecommenderSystems}.

Accepted at CIKM2019

References in corpus (3)

• xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
• NAIS: Neural Attentive Item Similarity Model for Recommendation
• Session-based Social Recommendation via Dynamic Graph Attention Networks

Cited by in corpus (64)

• DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
• Filter-enhanced MLP is All You Need for Sequential Recommendation
• DisenHAN: Disentangled Heterogeneous Graph Attention Network for Recommendation
• Disentangling Long and Short-Term Interests for Recommendation
• An Embedding Learning Framework for Numerical Features in CTR Prediction
• $F{M}^2$: Field-matrixed Factorization Machines for Recommender Systems
• CARCA: Context and Attribute-Aware Next-Item Recommendation via Cross-Attention
• CL4CTR: A Contrastive Learning Framework for CTR Prediction
• Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction
• EulerNet: Adaptive Feature Interaction Learning via Euler's Formula for CTR Prediction
• Learning Graph Meta Embeddings for Cold-Start Ads in Click-Through Rate Prediction
• Neural Graph Matching based Collaborative Filtering
• Enhancing CTR Prediction with Context-Aware Feature Representation Learning
• Deep Interest Highlight Network for Click-Through Rate Prediction in Trigger-Induced Recommendation
• Network On Network for Tabular Data Classification in Real-world Applications
• TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
• OptEmbed: Learning Optimal Embedding Table for Click-through Rate Prediction
• Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction
• Detecting Arbitrary Order Beneficial Feature Interactions for Recommender Systems
• DVR: Micro-Video Recommendation Optimizing Watch-Time-Gain under Duration Bias
• Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation
• Ads Recommendation in a Collapsed and Entangled World
• Uncovering User Interest from Biased and Noised Watch Time in Video Recommendation
• Edge-cloud Collaborative Learning with Federated and Centralized Features
• Looking at CTR Prediction Again: Is Attention All You Need?
• NASRec: Weight Sharing Neural Architecture Search for Recommender Systems
• A General Method For Automatic Discovery of Powerful Interactions In Click-Through Rate Prediction
• HIEN: Hierarchical Intention Embedding Network for Click-Through Rate Prediction
• MvFS: Multi-view Feature Selection for Recommender System
• CT4Rec: Simple yet Effective Consistency Training for Sequential Recommendation
• DCAP: Deep Cross Attentional Product Network for User Response Prediction
• Collaborative-Enhanced Prediction of Spending on Newly Downloaded Mobile Games under Consumption Uncertainty
• A Scalable Framework for Automatic Playlist Continuation on Music Streaming Services
• FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Data
• Unbiased Delayed Feedback Label Correction for Conversion Rate Prediction
• MV-HAN: A Hybrid Attentive Networks based Multi-View Learning Model for Large-scale Contents Recommendation
• Deep Evolutional Instant Interest Network for CTR Prediction in Trigger-Induced Recommendation
• Learning and Optimization of Implicit Negative Feedback for Industrial Short-video Recommender System
• FiBiNet++: Reducing Model Size by Low Rank Feature Interaction Layer for CTR Prediction
• A Federated Learning Benchmark on Tabular Data: Comparing Tree-Based Models and Neural Networks
• Cognitive Representation Learning of Self-Media Online Article Quality
• GraphFM: Graph Factorization Machines for Feature Interaction Modeling
• Local Contrastive Feature learning for Tabular Data
• SSIN: Self-Supervised Learning for Rainfall Spatial Interpolation
• AIE: Auction Information Enhanced Framework for CTR Prediction in Online Advertising
• Pareto Pairwise Ranking for Fairness Enhancement of Recommender Systems
• FedUD: Exploiting Unaligned Data for Cross-Platform Federated Click-Through Rate Prediction
• DLF: Enhancing Explicit-Implicit Interaction via Dynamic Low-Order-Aware Fusion for CTR Prediction
• Retrieval-Oriented Knowledge for Click-Through Rate Prediction
• ELEC: Efficient Large Language Model-Empowered Click-Through Rate Prediction
• Towards Better Modeling with Missing Data: A Contrastive Learning-based Visual Analytics Perspective
• Deep Unified Representation for Heterogeneous Recommendation
• Adaptive Learning on User Segmentation: Universal to Specific Representation via Bipartite Neural Interaction
• GACE: Learning Graph-Based Cross-Page Ads Embedding For Click-Through Rate Prediction
• Retrieval with Learned Similarities
• Toward a benchmark for CTR prediction in online advertising: datasets, evaluation protocols and perspectives
• Can Offline Metrics Measure Explanation Goals? A Comparative Survey Analysis of Offline Explanation Metrics in Recommender Systems
• InteractRank: Personalized Web-Scale Search Pre-Ranking with Cross Interaction Features
• Distribution-Guided Auto-Encoder for User Multimodal Interest Cross Fusion
• TabKAN: Advancing Tabular Data Analysis using Kolmogorov-Arnold Network
• MemoNet: Memorizing All Cross Features' Representations Efficiently via Multi-Hash Codebook Network for CTR Prediction
• RAMP: Robust Ad Recommendation Under Limited Personalized-Feature Availability via Masking and Alignment Pathways
• ORCA: Mitigating Over-Reliance for Multi-Task Dwell Time Prediction with Causal Decoupling
• Dual-Stream MLP is All You Need for CTR Prediction