MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation

Abstract

The goal of sequential recommendation (SR) is to predict a user's potential interested items based on her/his historical interaction sequences. Most existing sequential recommenders are developed based on ID features, which, despite their widespread use, often underperform with sparse IDs and struggle with the cold-start problem. Besides, inconsistent ID mappings hinder the model's transferability, isolating similar recommendation domains that could have been co-optimized. This paper aims to address these issues by exploring the potential of multi-modal information in learning robust and generalizable sequence representations. We propose MISSRec, a multi-modal pre-training and transfer learning framework for SR. On the user side, we design a Transformer-based encoder-decoder model, where the contextual encoder learns to capture the sequence-level multi-modal user interests while a novel interest-aware decoder is developed to grasp item-modality-interest relations for better sequence representation. On the candidate item side, we adopt a dynamic fusion module to produce user-adaptive item representation, providing more precise matching between users and items. We pre-train the model with contrastive learning objectives and fine-tune it in an efficient manner. Extensive experiments demonstrate the effectiveness and flexibility of MISSRec, promising a practical solution for real-world recommendation scenarios. Data and code are available on \url{https://github.com/gimpong/MM23-MISSRec}.

arXiv commentAccepted to ACM MM 2023. Data and code are available

References in corpus (6)

• Causal Intervention for Leveraging Popularity Bias in Recommendation
• Intent Contrastive Learning for Sequential Recommendation
• Sequential Recommender Systems: Challenges, Progress and Prospects
• Mining Latent Structures for Multimedia Recommendation
• Bootstrap Latent Representations for Multi-modal Recommendation
• Click-Through Rate Prediction with Multi-Modal Hypergraphs

Cited by in corpus (7)

• Exploring Adapter-based Transfer Learning for Recommender Systems: Empirical Studies and Practical Insights
• IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT
• Embedding Compression in Recommender Systems: A Survey
• Multi-Modal Multi-Behavior Sequential Recommendation with Conditional Diffusion-Based Feature Denoising
• A Pre-trained Sequential Recommendation Framework: Popularity Dynamics for Zero-shot Transfer
• RAT: Retrieval-Augmented Transformer for Click-Through Rate Prediction
• Intrinsic and Extrinsic Factor Disentanglement for Recommendation in Various Context Scenarios