Adap-$\tau$: Adaptively Modulating Embedding Magnitude for Recommendation

Abstract

Recent years have witnessed the great successes of embedding-based methods in recommender systems. Despite their decent performance, we argue one potential limitation of these methods -- the embedding magnitude has not been explicitly modulated, which may aggravate popularity bias and training instability, hindering the model from making a good recommendation. It motivates us to leverage the embedding normalization in recommendation. By normalizing user/item embeddings to a specific value, we empirically observe impressive performance gains (9\% on average) on four real-world datasets. Although encouraging, we also reveal a serious limitation when applying normalization in recommendation -- the performance is highly sensitive to the choice of the temperature $\tau$ which controls the scale of the normalized embeddings. To fully foster the merits of the normalization while circumvent its limitation, this work studied on how to adaptively set the proper $\tau$. Towards this end, we first make a comprehensive analyses of $\tau$ to fully understand its role on recommendation. We then accordingly develop an adaptive fine-grained strategy Adap-$\tau$ for the temperature with satisfying four desirable properties including adaptivity, personalized, efficiency and model-agnostic. Extensive experiments have been conducted to validate the effectiveness of the proposal. The code is available at \url{https://github.com/junkangwu/Adap_tau}.

References in corpus (8)

• Neural Graph Collaborative Filtering
• Graph Convolutional Neural Networks for Web-Scale Recommender Systems
• Self-supervised Graph Learning for Recommendation
• NormFace: L2 Hypersphere Embedding for Face Verification
• DeepInf: Social Influence Prediction with Deep Learning
• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning
• Embarrassingly Shallow Autoencoders for Sparse Data
• Embedding-based Retrieval in Facebook Search

Cited by in corpus (6)

• A Survey on Popularity Bias in Recommender Systems
• Large Language Models are Learnable Planners for Long-Term Recommendation
• Intersectional Two-sided Fairness in Recommendation
• Rankformer: A Graph Transformer for Recommendation based on Ranking Objective
• ESANS: Effective and Semantic-Aware Negative Sampling for Large-Scale Retrieval Systems
• On Inherited Popularity Bias in Cold-Start Item Recommendation