Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction

Abstract

Click-through rate (CTR) prediction is critical for industrial applications such as recommender system and online advertising. Practically, it plays an important role for CTR modeling in these applications by mining user interest from rich historical behavior data. Driven by the development of deep learning, deep CTR models with ingeniously designed architecture for user interest modeling have been proposed, bringing remarkable improvement of model performance over offline metric.However, great efforts are needed to deploy these complex models to online serving system for realtime inference, facing massive traffic request. Things turn to be more difficult when it comes to long sequential user behavior data, as the system latency and storage cost increase approximately linearly with the length of user behavior sequence. In this paper, we face directly the challenge of long sequential user behavior modeling and introduce our hands-on practice with the co-design of machine learning algorithm and online serving system for CTR prediction task. Theoretically, the co-design solution of UIC and MIMN enables us to handle the user interest modeling with unlimited length of sequential behavior data. Comparison between model performance and system efficiency proves the effectiveness of proposed solution. To our knowledge, this is one of the first industrial solutions that are capable of handling long sequential user behavior data with length scaling up to thousands. It now has been deployed in the display advertising system in Alibaba.

9 pages. Accepted by KDD 2019

References in corpus (3)

• Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks
• Learning Tree-based Deep Model for Recommender Systems
• Collaborative Memory Network for Recommendation Systems

Cited by in corpus (34)

• Click-Through Rate Prediction in Online Advertising: A Literature Review
• KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos
• Disentangling Long and Short-Term Interests for Recommendation
• User Behavior Retrieval for Click-Through Rate Prediction
• ATBRG: Adaptive Target-Behavior Relational Graph Network for Effective Recommendation
• Multi-Task Fusion via Reinforcement Learning for Long-Term User Satisfaction in Recommender Systems
• Click-Through Rate Prediction with Multi-Modal Hypergraphs
• Deep Interest Highlight Network for Click-Through Rate Prediction in Trigger-Induced Recommendation
• TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
• What is Next when Sequential Prediction Meets Implicitly Hard Interaction?
• Triangle Graph Interest Network for Click-through Rate Prediction
• HIEN: Hierarchical Intention Embedding Network for Click-Through Rate Prediction
• Deep Pattern Network for Click-Through Rate Prediction
• Rethinking Personalized Ranking at Pinterest: An End-to-End Approach
• MTBRN: Multiplex Target-Behavior Relation Enhanced Network for Click-Through Rate Prediction
• Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Meta
• Sparse Attentive Memory Network for Click-through Rate Prediction with Long Sequences
• Optimizing E-commerce Search: Toward a Generalizable and Rank-Consistent Pre-Ranking Model
• Modeling User Fatigue for Sequential Recommendation
• Graph Based Long-Term And Short-Term Interest Model for Click-Through Rate Prediction
• Temporal Interest Network for User Response Prediction
• Context-based Fast Recommendation Strategy for Long User Behavior Sequence in Meituan Waimai
• A Framework for Elastic Adaptation of User Multiple Intents in Sequential Recommendation
• Deep Situation-Aware Interaction Network for Click-Through Rate Prediction
• Exploring Scaling Laws of CTR Model for Online Performance Improvement
• Scaling Transformers for Discriminative Recommendation via Generative Pretraining
• A Centralized-Distributed Transfer Model for Cross-Domain Recommendation Based on Multi-Source Heterogeneous Transfer Learning
• Taming the One-Epoch Phenomenon in Online Recommendation System by Two-stage Contrastive ID Pre-training
• Deep Multiple Quantization Network on Long Behavior Sequence for Click-Through Rate Prediction
• ENCODE: Breaking the Trade-Off Between Performance and Efficiency in Long-Term User Behavior Modeling
• Retrieval-Oriented Knowledge for Click-Through Rate Prediction
• Modeling Long-term User Behaviors with Diffusion-driven Multi-interest Network for CTR Prediction
• Distribution-Guided Auto-Encoder for User Multimodal Interest Cross Fusion
• LREA: Low-Rank Efficient Attention on Modeling Long-Term User Behaviors for CTR Prediction