Lower-Left Partial AUC: An Effective and Efficient Optimization Metric for Recommendation

Abstract

Optimization metrics are crucial for building recommendation systems at scale. However, an effective and efficient metric for practical use remains elusive. While Top-K ranking metrics are the gold standard for optimization, they suffer from significant computational overhead. Alternatively, the more efficient accuracy and AUC metrics often fall short of capturing the true targets of recommendation tasks, leading to suboptimal performance. To overcome this dilemma, we propose a new optimization metric, Lower-Left Partial AUC (LLPAUC), which is computationally efficient like AUC but strongly correlates with Top-K ranking metrics. Compared to AUC, LLPAUC considers only the partial area under the ROC curve in the Lower-Left corner to push the optimization focus on Top-K. We provide theoretical validation of the correlation between LLPAUC and Top-K ranking metrics and demonstrate its robustness to noisy user feedback. We further design an efficient point-wise recommendation loss to maximize LLPAUC and evaluate it on three datasets, validating its effectiveness and robustness.

WWW 2024; 15 pages

References in corpus (8)

• BPR: Bayesian Personalized Ranking from Implicit Feedback
• A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation
• Denoising Implicit Feedback for Recommendation
• Alleviating Structural Distribution Shift in Graph Anomaly Detection
• On the Theories Behind Hard Negative Sampling for Recommendation
• On the Consistency of AUC Pairwise Optimization
• Optimizing Two-way Partial AUC with an End-to-end Framework
• CROLoss: Towards a Customizable Loss for Retrieval Models in Recommender Systems

Cited by in corpus (1)

• Breaking the Top-$K$ Barrier: Advancing Top-$K$ Ranking Metrics Optimization in Recommender Systems