Breaking the Top-$K$ Barrier: Advancing Top-$K$ Ranking Metrics Optimization in Recommender Systems

Abstract

In the realm of recommender systems (RS), Top-$K$ ranking metrics such as NDCG@$K$ are the gold standard for evaluating recommendation performance. However, during the training of recommendation models, optimizing NDCG@$K$ poses significant challenges due to its inherent discontinuous nature and the intricate Top-$K$ truncation. Recent efforts to optimize NDCG@$K$ have either overlooked the Top-$K$ truncation or suffered from high computational costs and training instability. To overcome these limitations, we propose SoftmaxLoss@$K$ (SL@$K$), a novel recommendation loss tailored for NDCG@$K$ optimization. Specifically, we integrate the quantile technique to handle Top-$K$ truncation and derive a smooth upper bound for optimizing NDCG@$K$ to address discontinuity. The resulting SL@$K$ loss has several desirable properties, including theoretical guarantees, ease of implementation, computational efficiency, gradient stability, and noise robustness. Extensive experiments on four real-world datasets and three recommendation backbones demonstrate that SL@$K$ outperforms existing losses with a notable average improvement of 6.03%. The code is available at https://github.com/Tiny-Snow/IR-Benchmark.

Accepted by KDD 2025

References in corpus (8)

• AutoDebias: Learning to Debias for Recommendation
• On the Effectiveness of Sampled Softmax Loss for Item Recommendation
• Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendation
• On Sampling Top-K Recommendation Evaluation
• On the Theories Behind Hard Negative Sampling for Recommendation
• Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls and New Benchmarking
• How Do Recommendation Models Amplify Popularity Bias? An Analysis from the Spectral Perspective
• Lower-Left Partial AUC: An Effective and Efficient Optimization Metric for Recommendation