Flexible distribution-free conditional predictive bands using density estimators

Abstract

Conformal methods create prediction bands that control average coverage under no assumptions besides i.i.d. data. Besides average coverage, one might also desire to control conditional coverage, that is, coverage for every new testing point. However, without strong assumptions, conditional coverage is unachievable. Given this limitation, the literature has focused on methods with asymptotical conditional coverage. In order to obtain this property, these methods require strong conditions on the dependence between the target variable and the features. We introduce two conformal methods based on conditional density estimators that do not depend on this type of assumption to obtain asymptotic conditional coverage: Dist-split and CD-split. While Dist-split asymptotically obtains optimal intervals, which are easier to interpret than general regions, CD-split obtains optimal size regions, which are smaller than intervals. CD-split also obtains local coverage by creating a data-driven partition of the feature space that scales to high-dimensional settings and by generating prediction bands locally on the partition elements. In a wide variety of simulated scenarios, our methods have a better control of conditional coverage and have smaller length than previously proposed methods.

References in corpus (9)

• Masked Autoregressive Flow for Density Estimation
• Flexible statistical inference for mechanistic models of neural dynamics
• Nonparametric Conditional Density Estimation in a High-Dimensional Regression Setting
• Conformalized Quantile Regression
• Conditional Density Estimation Tools in Python and R with Applications to Photometric Redshifts and Likelihood-Free Cosmological Inference
• A Unified Framework for Constructing, Tuning and Assessing Photometric Redshift Density Estimates in a Selection Bias Setting
• Conformal prediction with localization
• Conditional validity of inductive conformal predictors
• (f)RFCDE: Random Forests for Conditional Density Estimation and Functional Data

Cited by in corpus (8)

• Testing for Outliers with Conformal p-values
• Nested conformal prediction and quantile out-of-bag ensemble methods
• Conformal prediction with localization
• Improved conformalized quantile regression
• Conformal Prediction using Conditional Histograms
• Private Prediction Sets
• Exchangeability, Conformal Prediction, and Rank Tests
• Adaptive Conformal Prediction by Reweighting Nonconformity Score