Exclusivity Classes and Partitions of Loss Functions

Abstract

Loss functions define estimator optimality, yet current decision-theoretic tools say little about when different losses demand incompatible optimal procedures. This paper introduces a general framework for such incompatibilities using exclusivity regions, classes, and partitions of loss spaces relative to an abstract optimality operator. These partitions decompose a loss family into regimes such that no single estimator can be optimal across distinct regimes. We develop their basic structure, including links to conic geometry and invariance under positive scaling. The framework is formalized for quantile losses, convex margin-based classification losses, and the Huber robust-regression family on skewed models. Together with collapse results, the theory becomes a calculus of loss-design relevance: it identifies which features of a loss can affect the optimal estimator and which cannot. It yields no-free-lunch results for distinct quantile levels, robustness thresholds, and margin invariants, while also showing irrelevance results such as the fact that all classification-calibrated convex surrogates induce the same Bayes classifier. Applications include robust regression, logistic losses, elicitation theory, and model-robust loss partitions.