Selecting informative conformal prediction sets with false coverage rate control

Abstract

In supervised learning, including regression and classification, conformal methods provide prediction sets for the outcome/label with finite sample coverage for any machine learning predictor. We consider here the case where such prediction sets come after a selection process. The selection process requires that the selected prediction sets be `informative' in a well defined sense. We consider both the classification and regression settings where the analyst may consider as informative only the sample with prediction sets small enough, excluding null values, or obeying other appropriate `monotone' constraints. We develop a unified framework for building such informative conformal prediction sets while controlling the false coverage rate (FCR) on the selected sample. While conformal prediction sets after selection have been the focus of much recent literature in the field, the new introduced procedures, called InfoSP and InfoSCOP, are to our knowledge the first ones providing FCR control for informative prediction sets. We show the usefulness of our resulting procedures on real and simulated data.

50 pages, 15 figures, 1 table. Journal of the Royal Statistical Society Series B: Statistical Methodology, 2025

References in corpus (11)

• Least Ambiguous Set-Valued Classifiers with Bounded Error Levels
• Testing for Outliers with Conformal p-values
• Nested conformal prediction and quantile out-of-bag ensemble methods
• Class-Conditional Conformal Prediction with Many Classes
• Selection by Prediction with Conformal p-values
• Selective conformal inference with false coverage-statement rate control
• A Burden Shared is a Burden Halved: A Fairness-Adjusted Approach to Classification
• Confidence on the Focal: Conformal Prediction with Selection-Conditional Coverage
• False membership rate control in mixture models
• Controlling FSR in Selective Classification
• A constructive approach to selective risk control