Conformal Prediction using Conditional Histograms

Abstract

This paper develops a conformal method to compute prediction intervals for non-parametric regression that can automatically adapt to skewed data. Leveraging black-box machine learning algorithms to estimate the conditional distribution of the outcome using histograms, it translates their output into the shortest prediction intervals with approximate conditional coverage. The resulting prediction intervals provably have marginal coverage in finite samples, while asymptotically achieving conditional coverage and optimal length if the black-box model is consistent. Numerical experiments with simulated and real data demonstrate improved performance compared to state-of-the-art alternatives, including conformalized quantile regression and other distributional conformal prediction approaches.

arXiv comment12 pages, 4 figures. Supplement: 15 pages, 3 figures, 1 table

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• Testing for Outliers with Conformal p-values
• Nested conformal prediction and quantile out-of-bag ensemble methods
• Nonparametric Conditional Density Estimation in a High-Dimensional Regression Setting
• Adaptive, Distribution-Free Prediction Intervals for Deep Networks
• Flexible distribution-free conditional predictive bands using density estimators
• Making learning more transparent using conformalized performance prediction

Cited by in corpus (5)

• A Two-Sample Conditional Distribution Test Using Conformal Prediction and Weighted Rank Sum
• Improved conformalized quantile regression
• Dual-Splitting Conformal Prediction for Multi-Step Time Series Forecasting
• Conformalised data synthesis
• Bin-Conditional Conformal Prediction of Fatalities from Armed Conflict