An Unified Search and Recommendation Foundation Model for Cold-Start Scenario

Abstract

In modern commercial search engines and recommendation systems, data from multiple domains is available to jointly train the multi-domain model. Traditional methods train multi-domain models in the multi-task setting, with shared parameters to learn the similarity of multiple tasks, and task-specific parameters to learn the divergence of features, labels, and sample distributions of individual tasks. With the development of large language models, LLM can extract global domain-invariant text features that serve both search and recommendation tasks. We propose a novel framework called S\&R Multi-Domain Foundation, which uses LLM to extract domain invariant features, and Aspect Gating Fusion to merge the ID feature, domain invariant text features and task-specific heterogeneous sparse features to obtain the representations of query and item. Additionally, samples from multiple search and recommendation scenarios are trained jointly with Domain Adaptive Multi-Task module to obtain the multi-domain foundation model. We apply the S\&R Multi-Domain foundation model to cold start scenarios in the pretrain-finetune manner, which achieves better performance than other SOTA transfer learning methods. The S\&R Multi-Domain Foundation model has been successfully deployed in Alipay Mobile Application's online services, such as content query recommendation and service card recommendation, etc.

arXiv commentCIKM 2023,6 pages

References in corpus (10)

• LLaMA: Open and Efficient Foundation Language Models
• A Survey of Large Language Models
• GLM-130B: An Open Bilingual Pre-trained Model
• A Model-Agnostic Causal Learning Framework for Recommendation using Search Data
• When Search Meets Recommendation: Learning Disentangled Search Representation for Recommendation
• M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems
• USER: A Unified Information Search and Recommendation Model based on Integrated Behavior Sequence
• One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction
• Infer Implicit Contexts in Real-time Online-to-Offline Recommendation
• Prototypical Contrastive Learning and Adaptive Interest Selection for Candidate Generation in Recommendations

Cited by in corpus (1)

• A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios