On the Effectiveness of Sampled Softmax Loss for Item Recommendation

Abstract

The learning objective plays a fundamental role to build a recommender system. Most methods routinely adopt either pointwise or pairwise loss to train the model parameters, while rarely pay attention to softmax loss due to its computational complexity when scaling up to large datasets or intractability for streaming data. The sampled softmax (SSM) loss emerges as an efficient substitute for softmax loss. Its special case, InfoNCE loss, has been widely used in self-supervised learning and exhibited remarkable performance for contrastive learning. Nonetheless, limited recommendation work uses the SSM loss as the learning objective. Worse still, none of them explores its properties thoroughly and answers ``Does SSM loss suit for item recommendation?'' and ``What are the conceptual advantages of SSM loss, as compared with the prevalent losses?'', to the best of our knowledge. In this work, we aim to offer a better understanding of SSM for item recommendation. Specifically, we first theoretically reveal three model-agnostic advantages: (1) mitigating popularity bias; (2) mining hard negative samples; and (3) maximizing the ranking metric. However, based on our empirical studies, we recognize that the default choice of cosine similarity function in SSM limits its ability in learning the magnitudes of representation vectors. As such, the combinations of SSM with the models that also fall short in adjusting magnitudes may result in poor representations. One step further, we provide mathematical proof that message passing schemes in graph convolution networks can adjust representation magnitude according to node degree, which naturally compensates for the shortcoming of SSM. Extensive experiments on four benchmark datasets justify our analyses, demonstrating the superiority of SSM for item recommendation. Our implementations are available in both TensorFlow and PyTorch.

Accepted by TOIS

References in corpus (4)

• Self-supervised Graph Learning for Recommendation
• S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization
• Causal Intervention for Leveraging Popularity Bias in Recommendation
• GIF: A General Graph Unlearning Strategy via Influence Function

Cited by in corpus (15)

• Neighborhood-Enhanced Supervised Contrastive Learning for Collaborative Filtering
• Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learning
• How Do Recommendation Models Amplify Popularity Bias? An Analysis from the Spectral Perspective
• Que2Engage: Embedding-based Retrieval for Relevant and Engaging Products at Facebook Marketplace
• GradCraft: Elevating Multi-task Recommendations through Holistic Gradient Crafting
• Negative Feedback for Music Personalization
• Rankformer: A Graph Transformer for Recommendation based on Ranking Objective
• Scaling Transformers for Discriminative Recommendation via Generative Pretraining
• Breaking the Top-$K$ Barrier: Advancing Top-$K$ Ranking Metrics Optimization in Recommender Systems
• eSASRec: Enhancing Transformer-based Recommendations in a Modular Fashion
• Evaluating and Addressing Fairness Across User Groups in Negative Sampling for Recommender Systems
• Correcting the LogQ Correction: Revisiting Sampled Softmax for Large-Scale Retrieval
• Exploiting Preferences in Loss Functions for Sequential Recommendation via Weak Transitivity
• Simple and Powerful Architecture for Inductive Recommendation Using Knowledge Graph Convolutions
• RecPFN: Prior-Fitted Networks for In-Context-Based Recommendations