Predicting accurate probabilities with a ranking loss

Abstract

In many real-world applications of machine learning classifiers, it is essential to predict the probability of an example belonging to a particular class. This paper proposes a simple technique for predicting probabilities based on optimizing a ranking loss, followed by isotonic regression. This semi-parametric technique offers both good ranking and regression performance, and models a richer set of probability distributions than statistical workhorses such as logistic regression. We provide experimental results that show the effectiveness of this technique on real-world applications of probability prediction.

ICML2012

Cited by in corpus (11)

• Estimating treatment effect heterogeneity in randomized program evaluation
• Thresholding Classifiers to Maximize F1 Score
• Deep Learning for Patient-Specific Kidney Graft Survival Analysis
• Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labels
• Adversarial Time-to-Event Modeling
• Optimizing Two-way Partial AUC with an End-to-end Framework
• Calibrating sufficiently
• Online Isotonic Regression
• Intelligence, physics and information -- the tradeoff between accuracy and simplicity in machine learning
• Binary Classifier Calibration: Non-parametric approach
• $L_p$ Isotonic Regression Algorithms Using an $L_0$ Approach