S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization

Abstract

Recently, significant progress has been made in sequential recommendation with deep learning. Existing neural sequential recommendation models usually rely on the item prediction loss to learn model parameters or data representations. However, the model trained with this loss is prone to suffer from data sparsity problem. Since it overemphasizes the final performance, the association or fusion between context data and sequence data has not been well captured and utilized for sequential recommendation. To tackle this problem, we propose the model S^3-Rec, which stands for Self-Supervised learning for Sequential Recommendation, based on the self-attentive neural architecture. The main idea of our approach is to utilize the intrinsic data correlation to derive self-supervision signals and enhance the data representations via pre-training methods for improving sequential recommendation. For our task, we devise four auxiliary self-supervised objectives to learn the correlations among attribute, item, subsequence, and sequence by utilizing the mutual information maximization (MIM) principle. MIM provides a unified way to characterize the correlation between different types of data, which is particularly suitable in our scenario. Extensive experiments conducted on six real-world datasets demonstrate the superiority of our proposed method over existing state-of-the-art methods, especially when only limited training data is available. Besides, we extend our self-supervised learning method to other recommendation models, which also improve their performance.

Accepted as CIKM2020 long paper

References in corpus (1)

• Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks

Cited by in corpus (72)

• A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation
• Intent Contrastive Learning for Sequential Recommendation
• Filter-enhanced MLP is All You Need for Sequential Recommendation
• Multi-Modal Self-Supervised Learning for Recommendation
• Knowledge Graph Self-Supervised Rationalization for Recommendation
• Debiased Contrastive Learning for Sequential Recommendation
• KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems
• Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer
• Double-Scale Self-Supervised Hypergraph Learning for Group Recommendation
• Large Language Models as Zero-Shot Conversational Recommenders
• Disentangling Long and Short-Term Interests for Recommendation
• CauseRec: Counterfactual User Sequence Synthesis for Sequential Recommendation
• Lightweight Self-Attentive Sequential Recommendation
• Contrastive Cross-Domain Sequential Recommendation
• Knowledge Enhancement for Contrastive Multi-Behavior Recommendation
• Collaborative Large Language Model for Recommender Systems
• CARCA: Context and Attribute-Aware Next-Item Recommendation via Cross-Attention
• Graph Transformer for Recommendation
• Contrastive Collaborative Filtering for Cold-Start Item Recommendation
• Multi-Behavior Sequential Recommendation with Temporal Graph Transformer
• On the Effectiveness of Sampled Softmax Loss for Item Recommendation
• Modeling Two-Way Selection Preference for Person-Job Fit
• How to Index Item IDs for Recommendation Foundation Models
• AutoMLP: Automated MLP for Sequential Recommendations
• ConsRec: Learning Consensus Behind Interactions for Group Recommendation
• Click-Through Rate Prediction with Multi-Modal Hypergraphs
• gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling
• Dataset Regeneration for Sequential Recommendation
• Diffusion-based Contrastive Learning for Sequential Recommendation
• When Search Meets Recommendation: Learning Disentangled Search Representation for Recommendation
• Contrastive Multi-Level Graph Neural Networks for Session-based Recommendation
• Widespread Flaws in Offline Evaluation of Recommender Systems
• APGL4SR: A Generic Framework with Adaptive and Personalized Global Collaborative Information in Sequential Recommendation
• Ads Recommendation in a Collapsed and Entangled World
• What is Next when Sequential Prediction Meets Implicitly Hard Interaction?
• Scenario-Adaptive and Self-Supervised Model for Multi-Scenario Personalized Recommendation
• A Comprehensive Survey on Self-Supervised Learning for Recommendation
• Counterfactual Collaborative Reasoning
• Prompt-enhanced Federated Content Representation Learning for Cross-domain Recommendation
• Scaling Sequential Recommendation Models with Transformers
• Disentangling Past-Future Modeling in Sequential Recommendation via Dual Networks
• Towards Mitigating Dimensional Collapse of Representations in Collaborative Filtering
• EAGER-LLM: Enhancing Large Language Models as Recommenders through Exogenous Behavior-Semantic Integration
• On the Effectiveness of Unlearning in Session-Based Recommendation
• CT4Rec: Simple yet Effective Consistency Training for Sequential Recommendation
• Intent Disentanglement and Feature Self-supervision for Novel Recommendation
• FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Data
• Proxy-based Item Representation for Attribute and Context-aware Recommendation
• Collaboration and Transition: Distilling Item Transitions into Multi-Query Self-Attention for Sequential Recommendation
• Enhancing Sequential Music Recommendation with Personalized Popularity Awareness
• MUSE: Music Recommender System with Shuffle Play Recommendation Enhancement
• Mitigating Spurious Correlations for Self-supervised Recommendation
• Temporal Interest Network for User Response Prediction
• Consistency and Discrepancy-Based Contrastive Tripartite Graph Learning for Recommendations
• HMAR: Hierarchical Masked Attention for Multi-Behaviour Recommendation
• Enhancing Transformers without Self-supervised Learning: A Loss Landscape Perspective in Sequential Recommendation
• Filtering with Time-frequency Analysis: An Adaptive and Lightweight Model for Sequential Recommender Systems Based on Discrete Wavelet Transform
• Enhancing CTR Prediction through Sequential Recommendation Pre-training: Introducing the SRP4CTR Framework
• Deep Situation-Aware Interaction Network for Click-Through Rate Prediction
• Semantic-enhanced Co-attention Prompt Learning for Non-overlapping Cross-Domain Recommendation
• X-Cross: Dynamic Integration of Language Models for Cross-Domain Sequential Recommendation
• Enhancing Recommendation with Denoising Auxiliary Task
• eSASRec: Enhancing Transformer-based Recommendations in a Modular Fashion
• Separated Contrastive Learning for Matching in Cross-domain Recommendation with Curriculum Scheduling
• A Contextual-Aware Position Encoding for Sequential Recommendation
• EfficientRec an unlimited user-item scale recommendation system based on clustering and users interaction embedding profile
• Improving conversion rate prediction via self-supervised pre-training in online advertising
• Exploiting Preferences in Loss Functions for Sequential Recommendation via Weak Transitivity
• RUEL: Retrieval-Augmented User Representation with Edge Browser Logs for Sequential Recommendation
• RecPFN: Prior-Fitted Networks for In-Context-Based Recommendations
• SimDiffRec: Semantic Similarity-Guided Diffusion for Contrastive Sequential Recommendation
• MVIGER: Multi-View Variational Integration of Complementary Knowledge for Generative Recommender