Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction
arXiv:2101.03654 · doi:10.1145/3459637.3482088
Abstract
Click-Through Rate (CTR) prediction, whose aim is to predict the probability of whether a user will click on an item, is an essential task for many online applications. Due to the nature of data sparsity and high dimensionality of CTR prediction, a key to making effective prediction is to model high-order feature interaction. An efficient way to do this is to perform inner product of feature embeddings with self-attentive neural networks. To better model complex feature interaction, in this paper we propose a novel DisentanglEd Self-atTentIve NEtwork (DESTINE) framework for CTR prediction that explicitly decouples the computation of unary feature importance from pairwise interaction. Specifically, the unary term models the general importance of one feature on all other features, whereas the pairwise interaction term contributes to learning the pure impact for each feature pair. We conduct extensive experiments using two real-world benchmark datasets. The results show that DESTINE not only maintains computational efficiency but achieves consistent improvements over state-of-the-art baselines.
arXiv commentAccepted to CIKM 2021; authors' version
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Semi-Supervised Classification with Graph Convolutional Networks
- xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
- AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
- DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
- Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction
- GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond
- Neural Factorization Machines for Sparse Predictive Analytics
- Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction
- Higher-Order Factorization Machines
- TFNet: Multi-Semantic Feature Interaction for CTR Prediction