Selecting informative conformal prediction sets with false coverage rate control

Abstract

In supervised learning, including regression and classification, conformal methods provide prediction sets for the outcome/label with finite sample coverage for any machine learning predictor. We consider here the case where such prediction sets come after a selection process. The selection process requires that the selected prediction sets be `informative' in a well defined sense. We consider both the classification and regression settings where the analyst may consider as informative only the sample with prediction sets small enough, excluding null values, or obeying other appropriate `monotone' constraints. We develop a unified framework for building such informative conformal prediction sets while controlling the false coverage rate (FCR) on the selected sample. While conformal prediction sets after selection have been the focus of much recent literature in the field, the new introduced procedures, called InfoSP and InfoSCOP, are to our knowledge the first ones providing FCR control for informative prediction sets. We show the usefulness of our resulting procedures on real and simulated data.

50 pages, 15 figures, 1 table. Journal of the Royal Statistical Society Series B: Statistical Methodology, 2025

References in corpus (9)

• Testing for Outliers with Conformal p-values
• Class-Conditional Conformal Prediction with Many Classes
• Selection by Prediction with Conformal p-values
• Selective conformal inference with false coverage-statement rate control
• Confidence on the Focal: Conformal Prediction with Selection-Conditional Coverage
• A Burden Shared is a Burden Halved: A Fairness-Adjusted Approach to Classification
• Controlling FSR in Selective Classification
• False membership rate control in mixture models
• A constructive approach to selective risk control