Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction

Abstract

User response prediction, which models the user preference w.r.t. the presented items, plays a key role in online services. With two-decade rapid development, nowadays the cumulated user behavior sequences on mature Internet service platforms have become extremely long since the user's first registration. Each user not only has intrinsic tastes, but also keeps changing her personal interests during lifetime. Hence, it is challenging to handle such lifelong sequential modeling for each individual user. Existing methodologies for sequential modeling are only capable of dealing with relatively recent user behaviors, which leaves huge space for modeling long-term especially lifelong sequential patterns to facilitate user modeling. Moreover, one user's behavior may be accounted for various previous behaviors within her whole online activity history, i.e., long-term dependency with multi-scale sequential patterns. In order to tackle these challenges, in this paper, we propose a Hierarchical Periodic Memory Network for lifelong sequential modeling with personalized memorization of sequential patterns for each user. The model also adopts a hierarchical and periodical updating mechanism to capture multi-scale sequential patterns of user interests while supporting the evolving user behavior logs. The experimental results over three large-scale real-world datasets have demonstrated the advantages of our proposed model with significant improvement in user response prediction performance against the state-of-the-arts.

SIGIR 2019. Reproducible codes and datasets: https://github.com/alimamarankgroup/HPMN

References in corpus (18)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Recurrent Neural Networks with Top-k Gains for Session-based Recommendations
• End-To-End Memory Networks
• Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
• DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
• Learning Tree-based Deep Model for Recommender Systems
• Collaborative Memory Network for Recommendation Systems
• Hierarchical Multiscale Recurrent Neural Networks
• Vista: A Visually, Socially, and Temporally-aware Model for Artistic Recommendation
• Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation
• Bidding Machine: Learning to Bid for Directly Optimizing Profits in Display Advertising
• Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction
• Learning Piece-wise Linear Models from Large Scale Data for Ad Click Prediction
• Next Item Recommendation with Self-Attention
• Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate
• Learning Multi-touch Conversion Attribution with Dual-attention Mechanisms for Online Advertising
• Towards Training Recurrent Neural Networks for Lifelong Learning

Cited by in corpus (35)

• Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendation
• User Behavior Retrieval for Click-Through Rate Prediction
• Deep Learning for Sequential Recommendation: Algorithms, Influential Factors, and Evaluations
• ATBRG: Adaptive Target-Behavior Relational Graph Network for Effective Recommendation
• Network On Network for Tabular Data Classification in Real-world Applications
• TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
• Modeling Dynamic User Preference via Dictionary Learning for Sequential Recommendation
• End-to-End User Behavior Retrieval in Click-Through RatePrediction Model
• HIEN: Hierarchical Intention Embedding Network for Click-Through Rate Prediction
• One Person, One Model, One World: Learning Continual User Representation without Forgetting
• MTBRN: Multiplex Target-Behavior Relation Enhanced Network for Click-Through Rate Prediction
• Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems
• User Response Prediction in Online Advertising
• Learning to Respond with Your Favorite Stickers: A Framework of Unifying Multi-Modality and User Preference in Multi-Turn Dialog
• Graph Based Long-Term And Short-Term Interest Model for Click-Through Rate Prediction
• Dynamic Memory based Attention Network for Sequential Recommendation
• Rethinking Lifelong Sequential Recommendation with Incremental Multi-Interest Attention
• Temporal Interest Network for User Response Prediction
• Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction
• Context-based Fast Recommendation Strategy for Long User Behavior Sequence in Meituan Waimai
• Exploring Scaling Laws of CTR Model for Online Performance Improvement
• MOI-Mixer: Improving MLP-Mixer with Multi Order Interactions in Sequential Recommendation
• Deep Situation-Aware Interaction Network for Click-Through Rate Prediction
• Open Benchmarking for Click-Through Rate Prediction
• Multi-Interest-Aware User Modeling for Large-Scale Sequential Recommendations
• A Framework for Elastic Adaptation of User Multiple Intents in Sequential Recommendation
• Memory Augmented Neural Model for Incremental Session-based Recommendation
• Understanding or Manipulation: Rethinking Online Performance Gains of Modern Recommender Systems
• MISS: Multi-Interest Self-Supervised Learning Framework for Click-Through Rate Prediction
• Modeling Long-term User Behaviors with Diffusion-driven Multi-interest Network for CTR Prediction
• Retrieval-Oriented Knowledge for Click-Through Rate Prediction
• ENCODE: Breaking the Trade-Off Between Performance and Efficiency in Long-Term User Behavior Modeling
• Personalized Adaptive Meta Learning for Cold-start User Preference Prediction
• Dual Graph enhanced Embedding Neural Network for CTR Prediction
• Retrieval & Interaction Machine for Tabular Data Prediction