Exploring Adapter-based Transfer Learning for Recommender Systems: Empirical Studies and Practical Insights

Abstract

Adapters, a plug-in neural network module with some tunable parameters, have emerged as a parameter-efficient transfer learning technique for adapting pre-trained models to downstream tasks, especially for natural language processing (NLP) and computer vision (CV) fields. Meanwhile, learning recommendation models directly from raw item modality features -- e.g., texts of NLP and images of CV -- can enable effective and transferable recommender systems (called TransRec). In view of this, a natural question arises: can adapter-based learning techniques achieve parameter-efficient TransRec with good performance? To this end, we perform empirical studies to address several key sub-questions. First, we ask whether the adapter-based TransRec performs comparably to TransRec based on standard full-parameter fine-tuning? does it hold for recommendation with different item modalities, e.g., textual RS and visual RS. If yes, we benchmark these existing adapters, which have been shown to be effective in NLP and CV tasks, in item recommendation tasks. Third, we carefully study several key factors for the adapter-based TransRec in terms of where and how to insert these adapters? Finally, we look at the effects of adapter-based TransRec by either scaling up its source training data or scaling down its target training data. Our paper provides key insights and practical guidance on unified & transferable recommendation -- a less studied recommendation scenario. We release our codes and other materials at: https://github.com/westlake-repl/Adapter4Rec/.

Accepted by WSDM2024

References in corpus (17)

• LoRA: Low-Rank Adaptation of Large Language Models
• TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation
• Multi-Modal Self-Supervised Learning for Recommendation
• LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
• LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
• Prompt Learning for News Recommendation
• MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation
• Recommender Systems with Generative Retrieval
• M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems
• Scattered or Connected? An Optimized Parameter-efficient Tuning Approach for Information Retrieval
• End-to-end Learnable Diversity-aware News Recommendation
• Prompt Tuning Large Language Models on Personalized Aspect Extraction for Recommendations
• Thoroughly Modeling Multi-domain Pre-trained Recommendation as Language
• A Content-Driven Micro-Video Recommendation Dataset at Scale
• NineRec: A Benchmark Dataset Suite for Evaluating Transferable Recommendation
• ID Embedding as Subtle Features of Content and Structure for Multimodal Recommendation
• Collaborative Word-based Pre-trained Item Representation for Transferable Recommendation

Cited by in corpus (8)

• Recommender Systems in the Era of Large Language Models (LLMs)
• When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities
• IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT
• Towards End-to-End Explainable Facial Action Unit Recognition via Vision-Language Joint Learning
• ABXI: Invariant Interest Adaptation for Task-Guided Cross-Domain Sequential Recommendation
• Efficient and Effective Adaptation of Multimodal Foundation Models in Sequential Recommendation
• X-Cross: Dynamic Integration of Language Models for Cross-Domain Sequential Recommendation
• Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights