Improved conformalized quantile regression

Abstract

Conformalized quantile regression is a procedure that inherits the advantages of conformal prediction and quantile regression. That is, we use quantile regression to estimate the true conditional quantile and then apply a conformal step on a calibration set to ensure marginal coverage. In this way, we get adaptive prediction intervals that account for heteroscedasticity. However, the aforementioned conformal step lacks adaptiveness as described in (Romano et al., 2019). To overcome this limitation, instead of applying a single conformal step after estimating conditional quantiles with quantile regression, we propose to cluster the explanatory variables weighted by their permutation importance with an optimized k-means and apply k conformal steps. To show that this improved version outperforms the classic version of conformalized quantile regression and is more adaptive to heteroscedasticity, we extensively compare the prediction intervals of both in open datasets.

arXiv comment11 pages, 10 figures

References in corpus (12)

• Scikit-learn: Machine Learning in Python
• Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
• A tutorial on conformal prediction
• Estimating conditional quantiles with the help of the pinball loss
• Conformal Prediction: a Unified Review of Theory and New Challenges
• Reliable Prediction Intervals with Regression Neural Networks
• Conformal Prediction Interval Estimations with an Application to Day-Ahead and Intraday Power Markets
• Valid prediction intervals for regression problems
• A comparison of some conformal quantile regression methods
• Flexible distribution-free conditional predictive bands using density estimators
• Conformal Prediction using Conditional Histograms
• Integrating Uncertainty Awareness into Conformalized Quantile Regression