xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems

Abstract

Combinatorial features are essential for the success of many commercial models. Manually crafting these features usually comes with high cost due to the variety, volume and velocity of raw data in web-scale systems. Factorization based models, which measure interactions in terms of vector product, can learn patterns of combinatorial features automatically and generalize to unseen features as well. With the great success of deep neural networks (DNNs) in various fields, recently researchers have proposed several DNN-based factorization model to learn both low- and high-order feature interactions. Despite the powerful ability of learning an arbitrary function from data, plain DNNs generate feature interactions implicitly and at the bit-wise level. In this paper, we propose a novel Compressed Interaction Network (CIN), which aims to generate feature interactions in an explicit fashion and at the vector-wise level. We show that the CIN share some functionalities with convolutional neural networks (CNNs) and recurrent neural networks (RNNs). We further combine a CIN and a classical DNN into one unified model, and named this new model eXtreme Deep Factorization Machine (xDeepFM). On one hand, the xDeepFM is able to learn certain bounded-degree feature interactions explicitly; on the other hand, it can learn arbitrary low- and high-order feature interactions implicitly. We conduct comprehensive experiments on three real-world datasets. Our results demonstrate that xDeepFM outperforms state-of-the-art models. We have released the source code of xDeepFM at \url{https://github.com/Leavingseason/xDeepFM}.

arXiv comment10 pages

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
• Neural Factorization Machines for Sparse Predictive Analytics
• Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks
• Product-based Neural Networks for User Response Prediction
• Higher-Order Factorization Machines
• Deep & Cross Network for Ad Click Predictions
• Deep Interest Network for Click-Through Rate Prediction
• Cross-Device User Matching Based on Massive Browse Logs: The Runner-Up Solution for the 2016 CIKM Cup

Cited by in corpus (194)

• KGAT: Knowledge Graph Attention Network for Recommendation
• Deep Neural Networks and Tabular Data: A Survey
• AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
• DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
• A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation
• Deep Learning Recommendation Model for Personalization and Recommendation Systems
• Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction
• Multi-level Cross-view Contrastive Learning for Knowledge-aware Recommender System
• IntentGC: a Scalable Graph Convolution Framework Fusing Heterogeneous Information for Recommendation
• Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction
• Click-Through Rate Prediction in Online Advertising: A Literature Review
• DGCN: Diversified Recommendation with Graph Convolutional Networks
• Learning to Warm Up Cold Item Embeddings for Cold-start Recommendation with Meta Scaling and Shifting Networks
• User Behavior Retrieval for Click-Through Rate Prediction
• Improving Knowledge-aware Recommendation with Multi-level Interactive Contrastive Learning
• Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems
• An Embedding Learning Framework for Numerical Features in CTR Prediction
• $F{M}^2$: Field-matrixed Factorization Machines for Recommender Systems
• Graph Convolution Machine for Context-aware Recommender System
• Modeling Users' Behavior Sequences with Hierarchical Explainable Network for Cross-domain Fraud Detection
• How to Retrain Recommender System? A Sequential Meta-Learning Method
• CL4CTR: A Contrastive Learning Framework for CTR Prediction
• Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction
• An Integrative Paradigm for Enhanced Stroke Prediction: Synergizing XGBoost and xDeepFM Algorithms
• Adversarial Multimodal Representation Learning for Click-Through Rate Prediction
• EulerNet: Adaptive Feature Interaction Learning via Euler's Formula for CTR Prediction
• HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework
• Learning Graph Meta Embeddings for Cold-Start Ads in Click-Through Rate Prediction
• DiffNet++: A Neural Influence and Interest Diffusion Network for Social Recommendation
• Enhancing CTR Prediction with Context-Aware Feature Representation Learning
• Recommender systems based on graph embedding techniques: A comprehensive review
• RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms
• ARM-Net: Adaptive Relation Modeling Network for Structured Data
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction
• Behavior Sequence Transformer for E-commerce Recommendation in Alibaba
• Network On Network for Tabular Data Classification in Real-world Applications
• CatGCN: Graph Convolutional Networks with Categorical Node Features
• Joint Optimization of Tree-based Index and Deep Model for Recommender Systems
• Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems
• Multimodality Invariant Learning for Multimedia-Based New Item Recommendation
• The Digital Transformation in Health: How AI Can Improve the Performance of Health Systems
• Understanding WeChat User Preferences and "Wow" Diffusion
• Information Theoretic Counterfactual Learning from Missing-Not-At-Random Feedback
• Modeling Dynamic User Preference via Dictionary Learning for Sequential Recommendation
• Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction
• Detecting Arbitrary Order Beneficial Feature Interactions for Recommender Systems
• Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation
• Ads Recommendation in a Collapsed and Entangled World
• Differentiable Neural Input Search for Recommender Systems
• STUaNet: Understanding uncertainty in spatiotemporal collective human mobility
• MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask
• Looking at CTR Prediction Again: Is Attention All You Need?
• NASRec: Weight Sharing Neural Architecture Search for Recommender Systems
• Memory-efficient Embedding for Recommendations
• End-to-End User Behavior Retrieval in Click-Through RatePrediction Model
• A General Method For Automatic Discovery of Powerful Interactions In Click-Through Rate Prediction
• Inductive Graph Transformer for Delivery Time Estimation
• FLEN: Leveraging Field for Scalable CTR Prediction
• GateNet: Gating-Enhanced Deep Network for Click-Through Rate Prediction
• MTGR: Industrial-Scale Generative Recommendation Framework in Meituan
• One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction
• Discovering and Explaining the Representation Bottleneck of DNNs
• HIEN: Hierarchical Intention Embedding Network for Click-Through Rate Prediction
• Controllable Multi-Interest Framework for Recommendation
• Deep Pattern Network for Click-Through Rate Prediction
• Infer Implicit Contexts in Real-time Online-to-Offline Recommendation
• CT4Rec: Simple yet Effective Consistency Training for Sequential Recommendation
• Mixture of Virtual-Kernel Experts for Multi-Objective User Profile Modeling
• Diffusion-based Multi-modal Synergy Interest Network for Click-through Rate Prediction
• DCAP: Deep Cross Attentional Product Network for User Response Prediction
• FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine
• Explicit Feature Interaction-aware Graph Neural Networks
• Collaborative-Enhanced Prediction of Spending on Newly Downloaded Mobile Games under Consumption Uncertainty
• A Practical Incremental Method to Train Deep CTR Models
• Understanding the Ranking Loss for Recommendation with Sparse User Feedback
• Personalizing Similar Product Recommendations in Fashion E-commerce
• Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach
• Adaptive Factorization Network: Learning Adaptive-Order Feature Interactions
• Sparse Attentive Memory Network for Click-through Rate Prediction with Long Sequences
• SAIN: Self-Attentive Integration Network for Recommendation
• User Response Prediction in Online Advertising
• Learning and Optimization of Implicit Negative Feedback for Industrial Short-video Recommender System
• Modeling Dynamic Attributes for Next Basket Recommendation
• Detecting Beneficial Feature Interactions for Recommender Systems
• FiBiNet++: Reducing Model Size by Low Rank Feature Interaction Layer for CTR Prediction
• Distributed Equivalent Substitution Training for Large-Scale Recommender Systems
• Multi View Spatial-Temporal Model for Travel Time Estimation
• LogicRec: Recommendation with Users' Logical Requirements
• AutoCross: Automatic Feature Crossing for Tabular Data in Real-World Applications
• Conversion Prediction Using Multi-task Conditional Attention Networks to Support the Creation of Effective Ad Creative
• Privacy-Aware Recommender Systems Challenge on Twitter's Home Timeline
• Deep Learning for Click-Through Rate Estimation
• GraphFM: Graph Factorization Machines for Feature Interaction Modeling
• Cross-GCN: Enhancing Graph Convolutional Network with $k$-Order Feature Interactions
• Price-aware Recommendation with Graph Convolutional Networks
• Collaborative Metric Learning with Memory Network for Multi-Relational Recommender Systems
• Temporal Interest Network for User Response Prediction
• DADNN: Multi-Scene CTR Prediction via Domain-Aware Deep Neural Network
• Time-based Sequence Model for Personalization and Recommendation Systems
• Adversarial Collaborative Filtering for Free
• Correct Normalization Matters: Understanding the Effect of Normalization On Deep Neural Network Models For Click-Through Rate Prediction
• Pairwise Interactive Graph Attention Network for Context-Aware Recommendation
• FedCTR: Federated Native Ad CTR Prediction with Multi-Platform User Behavior Data
• Graph Factorization Machines for Cross-Domain Recommendation
• Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction
• Training with Multi-Layer Embeddings for Model Reduction
• AutoLoss: Automated Loss Function Search in Recommendations
• RAT: Retrieval-Augmented Transformer for Click-Through Rate Prediction
• Context-based Fast Recommendation Strategy for Long User Behavior Sequence in Meituan Waimai
• TayFCS: Towards Light Feature Combination Selection for Deep Recommender Systems
• i-Razor: A Differentiable Neural Input Razor for Feature Selection and Dimension Search in DNN-Based Recommender Systems
• Hierarchical Gating Networks for Sequential Recommendation
• DNN2LR: Interpretation-inspired Feature Crossing for Real-world Tabular Data
• AutoFT: Automatic Fine-Tune for Parameters Transfer Learning in Click-Through Rate Prediction
• Exploring Scaling Laws of CTR Model for Online Performance Improvement
• Delayed Feedback Modeling for the Entire Space Conversion Rate Prediction
• Open Benchmarking for Click-Through Rate Prediction
• Deep Situation-Aware Interaction Network for Click-Through Rate Prediction
• Unified Dual-Intent Translation for Joint Modeling of Search and Recommendation
• DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving
• A Unified Search and Recommendation Framework Based on Multi-Scenario Learning for Ranking in E-commerce
• Multi-Interest-Aware User Modeling for Large-Scale Sequential Recommendations
• ContextNet: A Click-Through Rate Prediction Framework Using Contextual information to Refine Feature Embedding
• Denoising User-aware Memory Network for Recommendation
• BBE-LSWCM: A Bootstrapped Ensemble of Long and Short Window Clickstream Models
• A Novel User Representation Paradigm for Making Personalized Candidate Retrieval
• Feature Interaction based Neural Network for Click-Through Rate Prediction
• Iterative Boosting Deep Neural Networks for Predicting Click-Through Rate
• Heterogeneous-Temporal Graph Convolutional Networks: Make the Community Detection Much Better
• Multi-Graph Convolution Collaborative Filtering
• RiskOracle: A Minute-level Citywide Traffic Accident Forecasting Framework
• FINT: Field-aware INTeraction Neural Network For CTR Prediction
• DLF: Enhancing Explicit-Implicit Interaction via Dynamic Low-Order-Aware Fusion for CTR Prediction
• FedUD: Exploiting Unaligned Data for Cross-Platform Federated Click-Through Rate Prediction
• Field-Embedded Factorization Machines for Click-through rate prediction
• Implicit Feedback Deep Collaborative Filtering Product Recommendation System
• Random Offset Block Embedding Array (ROBE) for CriteoTB Benchmark MLPerf DLRM Model : 1000$\times$ Compression and 3.1$\times$ Faster Inference
• AEFE: Automatic Embedded Feature Engineering for Categorical Features
• R4: A Framework for Route Representation and Route Recommendation
• Deep Unified Representation for Heterogeneous Recommendation
• ELEC: Efficient Large Language Model-Empowered Click-Through Rate Prediction
• ENCODE: Breaking the Trade-Off Between Performance and Efficiency in Long-Term User Behavior Modeling
• TPG-DNN: A Method for User Intent Prediction Based on Total Probability Formula and GRU Loss with Multi-task Learning
• LARR: Large Language Model Aided Real-time Scene Recommendation with Semantic Understanding
• MISS: Multi-Interest Self-Supervised Learning Framework for Click-Through Rate Prediction
• Spatial-Temporal Deep Intention Destination Networks for Online Travel Planning
• Adaptive Learning on User Segmentation: Universal to Specific Representation via Bipartite Neural Interaction
• Generalized Embedding Machines for Recommender Systems
• CoSam: An Efficient Collaborative Adaptive Sampler for Recommendation
• Analysis of Psychographic Indicators via LIWC and Their Correlation with CTR for Instagram Ads
• Explainable Student Performance Prediction With Personalized Attention for Explaining Why A Student Fails
• Enhancing Factorization Machines with Generalized Metric Learning
• Multi-Interactive Attention Network for Fine-grained Feature Learning in CTR Prediction
• Distribution-Guided Auto-Encoder for User Multimodal Interest Cross Fusion
• Privileged Features Distillation at Taobao Recommendations
• Learning Effective and Efficient Embedding via an Adaptively-Masked Twins-based Layer
• InteractRank: Personalized Web-Scale Search Pre-Ranking with Cross Interaction Features
• Deep Time-Stream Framework for Click-Through Rate Prediction by Tracking Interest Evolution
• MemoNet: Memorizing All Cross Features' Representations Efficiently via Multi-Hash Codebook Network for CTR Prediction
• Toward a benchmark for CTR prediction in online advertising: datasets, evaluation protocols and perspectives
• DPAN: Dynamic Preference-based and Attribute-aware Network for Relevant Recommendations
• Memory Augmented Graph Neural Networks for Sequential Recommendation
• Learning Multi-granular Quantized Embeddings for Large-Vocab Categorical Features in Recommender Systems
• Leaf-FM: A Learnable Feature Generation Factorization Machine for Click-Through Rate Prediction
• GAIN: Graph Attention & Interaction Network for Inductive Semi-Supervised Learning over Large-scale Graphs
• Sequence-Aware Factorization Machines for Temporal Predictive Analytics
• Interaction as Interference: A Quantum-Inspired Aggregation Approach
• RAMP: Robust Ad Recommendation Under Limited Personalized-Feature Availability via Masking and Alignment Pathways
• Scene-wise Adaptive Network for Dynamic Cold-start Scenes Optimization in CTR Prediction
• Session-aware Item-combination Recommendation with Transformer Network
• A Recommendation System to Enhance Midwives' Capacities in Low-Income Countries
• AdSEE: Investigating the Impact of Image Style Editing on Advertisement Attractiveness
• COUPA: An Industrial Recommender System for Online to Offline Service Platforms
• Dual Attentive Sequential Learning for Cross-Domain Click-Through Rate Prediction
• ScaleFreeCTR: MixCache-based Distributed Training System for CTR Models with Huge Embedding Table
• Dual Graph enhanced Embedding Neural Network for CTR Prediction
• Online Interaction Detection for Click-Through Rate Prediction
• SHORING: Design Provable Conditional High-Order Interaction Network via Symbolic Testing
• AdnFM: An Attentive DenseNet based Factorization Machine for CTR Prediction
• Retrieval & Interaction Machine for Tabular Data Prediction
• Ensemble Knowledge Distillation for CTR Prediction
• Context-aware Tree-based Deep Model for Recommender Systems
• FIVES: Feature Interaction Via Edge Search for Large-Scale Tabular Data
• Dynamic Parameterized Network for CTR Prediction
• An Efficient Agreement Mechanism in CapsNets By Pairwise Product
• DNN2LR: Automatic Feature Crossing for Credit Scoring
• Interpretable Artificial Intelligence through the Lens of Feature Interaction
• Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling
• Efficient Click-Through Rate Prediction for Developing Countries via Tabular Learning
• XCrossNet: Feature Structure-Oriented Learning for Click-Through Rate Prediction
• Cross-Modal Alignment with Mixture Experts Neural Network for Intral-City Retail Recommendation
• Learning Feature Interactions with Lorentzian Factorization Machine
• Gated Attentive-Autoencoder for Content-Aware Recommendation
• Dual-Stream MLP is All You Need for CTR Prediction