Unachievable Region in Precision-Recall Space and Its Effect on Empirical Evaluation
arXiv:1206.4667
Abstract
Precision-recall (PR) curves and the areas under them are widely used to summarize machine learning results, especially for data sets exhibiting class skew. They are often used analogously to ROC curves and the area under ROC curves. It is known that PR curves vary as class skew changes. What was not recognized before this paper is that there is a region of PR space that is completely unachievable, and the size of this region depends only on the skew. This paper precisely characterizes the size of that region and discusses its implications for empirical evaluation methodology in machine learning.
ICML2012, fixed citations to use correct tech report number
Cited by in corpus (7)
- Using Machine Learning to Calibrate Storm-Scale Probabilistic Guidance of Severe Weather Hazards in the Warn-on-Forecast System
- Analyzing Factors Associated with Fatal Road Crashes: A Machine Learning Approach
- Momentum Accelerates the Convergence of Stochastic AUPRC Maximization
- A tree augmented naive Bayesian network experiment for breast cancer prediction
- Summarizing the performances of a background subtraction algorithm measured on several videos
- Population and Empirical PR Curves for Assessment of Ranking Algorithms
- Complex-Object Visual Inspection via Multiple Lighting Configurations