Ranking the best instances

Abstract

We formulate the local ranking problem in the framework of bipartite ranking where the goal is to focus on the best instances. We propose a methodology based on the construction of real-valued scoring functions. We study empirical risk minimization of dedicated statistics which involve empirical quantiles of the scores. We first state the problem of finding the best instances which can be cast as a classification problem with mass constraint. Next, we develop special performance measures for the local ranking problem which extend the Area Under an ROC Curve (AUC/AROC) criterion and describe the optimal elements of these new criteria. We also highlight the fact that the goal of ranking the best instances cannot be achieved in a stage-wise manner where first, the best instances would be tentatively identified and then a standard AUC criterion could be applied. Eventually, we state preliminary statistical results for the local ranking problem.

29 pages

References in corpus (1)

• Risk bounds for statistical learning

Cited by in corpus (14)

• Natural Language Processing (almost) from Scratch
• Top Rank Optimization in Linear Time
• An efficient reduction of ranking to classification
• Surrogate Regret Bounds for Bipartite Ranking via Strongly Proper Losses
• Learning Fair Scoring Functions: Bipartite Ranking under ROC-based Fairness Constraints
• Convex Calibration Dimension for Multiclass Loss Matrices
• Surrogate Functions for Maximizing Precision at the Top
• A Probabilistic Theory of Supervised Similarity Learning for Pointwise ROC Curve Optimization
• Optimising HEP parameter fits via Monte Carlo weight derivative regression
• A plug-in approach to maximising precision at the top and recall at the top
• Concentration Inequalities for Two-Sample Rank Processes with Application to Bipartite Ranking
• The column measure and Gradient-Free Gradient Boosting
• A review on ranking problems in statistical learning
• Predictive Value Generalization Bounds